-- Logs begin at Fri 2022-08-12 11:25:42 UTC, end at Fri 2022-08-12 12:41:20 UTC. -- Aug 12 11:59:43.709994 dv-cinder-797970-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 12 11:59:43.718895 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 12 11:59:43.721655 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 12 11:59:43 2022] *** Aug 12 11:59:43.721837 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 12 11:59:43.722000 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 12 11:59:43.722128 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: nodename: dv-cinder-797970-6-iscsi Aug 12 11:59:43.722128 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: machine: x86_64 Aug 12 11:59:43.722128 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: clock source: unix Aug 12 11:59:43.722128 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: pcre jit disabled Aug 12 11:59:43.722128 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: detected number of CPU cores: 4 Aug 12 11:59:43.722128 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: current working directory: / Aug 12 11:59:43.722128 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: detected binary path: /usr/bin/uwsgi-core Aug 12 11:59:43.722128 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: your processes number limit is 47680 Aug 12 11:59:43.722128 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: your memory page size is 4096 bytes Aug 12 11:59:43.722649 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: detected max file descriptor number: 2048 Aug 12 11:59:43.722649 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: lock engine: pthread robust mutexes Aug 12 11:59:43.722649 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: thunder lock: enabled Aug 12 11:59:43.722649 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 12 11:59:43.722649 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 12 11:59:43.763134 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: Python main interpreter initialized at 0x55e09c4229c0 Aug 12 11:59:43.763134 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: python threads support enabled Aug 12 11:59:43.763134 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: your server socket listen backlog is limited to 100 connections Aug 12 11:59:43.763134 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: your mercy for graceful operations on workers is 90 seconds Aug 12 11:59:43.763508 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: mapped 403077 bytes (393 KB) for 2 cores Aug 12 11:59:43.763582 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: *** Operational MODE: preforking *** Aug 12 11:59:43.763821 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: *** uWSGI is running in multiple interpreter mode *** Aug 12 11:59:43.763821 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: spawned uWSGI master process (pid: 81780) Aug 12 11:59:43.764023 dv-cinder-797970-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 12 11:59:43.765034 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: spawned uWSGI worker 1 (pid: 81793, cores: 1) Aug 12 11:59:43.765034 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: spawned uWSGI worker 2 (pid: 81794, cores: 1) Aug 12 11:59:43.765256 dv-cinder-797970-6-iscsi devstack@keystone.service[81780]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 12 11:59:49.299760 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 12 11:59:49.299760 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: from cryptography.utils import int_from_bytes Aug 12 11:59:49.300537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 12 11:59:49.300537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: from cryptography.utils import int_from_bytes Aug 12 11:59:49.300537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 12 11:59:49.300537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: from cryptography.utils import int_from_bytes Aug 12 11:59:49.301120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 12 11:59:49.301120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: from cryptography.utils import int_from_bytes Aug 12 11:59:49.350168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 12 11:59:49.350368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 12 11:59:49.350563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] command line args: [] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 12 11:59:49.350730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 12 11:59:49.350838 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 12 11:59:49.350914 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 12 11:59:49.351020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ================================================================================ {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 12 11:59:49.351102 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] command line args: [] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 12 11:59:49.351336 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] admin_token = **** {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.351418 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 12 11:59:49.351541 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ================================================================================ {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 12 11:59:49.351650 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] config_dir = [] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.351810 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] admin_token = **** {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.351893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.352062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] config_dir = [] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.352143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] config_source = [] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.352254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.352336 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] debug = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.352439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] config_source = [] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.352515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.352812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] debug = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.352925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.353056 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.353152 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.353250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.353356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.353456 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.353554 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.353734 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.353842 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.353978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] list_limit = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.354566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] list_limit = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.354685 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] log_config_append = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.354793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] log_config_append = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.354944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.355050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.355176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] log_dir = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.355276 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] log_dir = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.355380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] log_file = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.355497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] log_file = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.355621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.355783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.355900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.356079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.356199 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.356333 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.356460 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.356578 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.356713 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.356827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.356952 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.357064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.357183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.357302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.357415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.357532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.357650 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.357770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.357887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.357991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.358102 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.358218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.358329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.358448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.358568 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.358695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.358838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.359006 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.359318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.359442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.359551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] publish_errors = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.359707 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.359834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.359952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.360113 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.360283 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.360398 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.360562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] publish_errors = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.360705 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.360825 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.360937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] standard_threads = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.361068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.361192 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.361305 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.361422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.361539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.361655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.361775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] use_journal = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.361892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.362009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] use_json = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.362009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] use_stderr = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.362232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] standard_threads = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.362349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] use_syslog = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.362468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.362584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.362702 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.362813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.362924 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.363034 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.363153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] use_journal = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.363270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.363382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] use_json = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.363492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.363602 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] use_stderr = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.363713 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.363905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] use_syslog = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.364024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.364142 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 12 11:59:49.364254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.364372 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.364483 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.364605 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.364741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.364859 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.364976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.365093 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.365235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.365358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.365516 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.365630 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.365742 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.365930 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.366046 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.366236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.366351 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.366463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.366726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.366841 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.366970 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.367087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.367204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.367315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.367457 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.367574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.367727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.367850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.367999 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.368114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.368260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.368373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.368514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.368660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.368785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.368904 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.369017 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.369128 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.369247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.369370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.369488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.369601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.370262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.370380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.370570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.370733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.370969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.371084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.371248 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.371369 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.371513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.371635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.371768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.371883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.371994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.372104 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.372242 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.372355 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.372466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.372582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.372723 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.372836 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.372962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.373075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.373185 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.373301 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.373412 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.373527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.373647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.373765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.373876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.373986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.374095 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.374190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.374332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.374504 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.374664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.374861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.375001 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.375167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.375309 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.375424 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.375525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.375641 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.375752 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.375876 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.375876 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.376091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.376209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.376413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.376526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.376668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.376785 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.376917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.377031 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.377138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.377255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.377367 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.377479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.377601 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.377720 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.377822 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.377932 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.378051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.378167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.378287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.378401 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.378503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.378619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.378731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.378855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.378975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.379089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.379190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.379301 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.379417 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.379530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.379649 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.379762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.379873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.379985 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.380103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.380215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.380338 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.380450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.380562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.380721 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.380836 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.380937 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.381060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.381182 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.381295 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.381406 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.381517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.381629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.381748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.381860 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.381971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.382082 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.382200 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.382313 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.382500 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.382613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.382724 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.382835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.382946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.383057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.383176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.383290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.383401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.383513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.383632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.383756 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.383868 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.383992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.384104 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.384216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.384334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.384446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.384557 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.384693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.384808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.384921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.385032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.385144 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.385255 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.385379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.385493 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.385604 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.385777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.385899 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.386012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.386132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.386245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.386359 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.386470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.386582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.386694 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.386818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.386932 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.387044 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.387155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.387267 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.387379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.387498 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.387611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.387730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.387842 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.387954 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.388065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.388189 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.388302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.388415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.388527 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] auth.external = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.388683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.388803 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.388928 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.389042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.389156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.389156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.389366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.389366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] auth.password = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.389585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.389698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] auth.token = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.389810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.389921 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.390039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.390178 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.390300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.390414 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.390535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.390648 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.390761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] auth.external = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.390874 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.390986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.391099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.391219 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.391334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.391446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.391558 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] credential.caching = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.391672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] auth.password = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.391790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.391912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] auth.token = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.392025 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.392138 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.392251 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.392363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.392475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.392599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.392729 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.392855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.392969 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.393083 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.393196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.393317 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.393430 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.393542 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.393654 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.393767 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] credential.caching = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.393879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.394000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.394114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.394226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.394339 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.394451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.394451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.394678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.394793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.394906 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.395018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.395130 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.395250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.395364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] federation.caching = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.395476 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.395588 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.395700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.395814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.396016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.396138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.396138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.396359 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.396479 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.396594 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.396732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.396848 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.396967 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.397081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.397193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.397305 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.397425 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.397539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] federation.caching = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.397651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.397764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.397878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.398111 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity.caching = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.398225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.398337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.398455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.398568 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.398681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.398806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.398921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.399039 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.399152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.399306 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.399418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.399537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.399649 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.399769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.399881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.399993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.400103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.400222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.400335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity.caching = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.400446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.400556 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.400692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.400816 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.401102 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.401253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.401377 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.401623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.401743 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.401743 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.401969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.402086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.402207 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.402321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.402435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.402554 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.402676 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.402791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.402903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.403016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.403130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.403243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.403364 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.403478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.403591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.403703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.403816 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.403928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.404049 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.404163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.404276 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.404388 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.404500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.404622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.404752 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.404866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.404866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.405077 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.405189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.405315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.405431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.405544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.405544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.405755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.405868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.405992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.406106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.406219 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.406351 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.406464 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.406464 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.406751 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.406867 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.406979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.407098 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.407212 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.407325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.407445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.407564 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.407678 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.407789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.407917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.408036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.408158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.408271 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.408385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.408497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.408609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.408771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.408897 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.409012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.409123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.409235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.409347 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.409459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.409579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.409693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.409805 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.409917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.410029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.410140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.410261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.410374 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.410487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.410598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.410711 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.410822 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.410942 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.411055 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.411171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.411282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.411394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.411507 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.411628 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.411741 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.411853 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.411965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.412076 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.412188 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.412308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.412421 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.412533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.412664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.412778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.412890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.413014 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.413129 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.413241 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.413353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.413465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.413577 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.413698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.413811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.413924 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.414036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.414148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.414266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.414387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.414501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.414613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.414726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.414838 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.414956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.415077 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.415190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.415302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.415302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.415512 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.415624 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.415746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.415860 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.415972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.415972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.416183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.416294 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.416413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.417107 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.417225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.417225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.417438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.417550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.417674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.417790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.417902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.418015 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.418127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.418247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.418367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.418480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.418594 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.418706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.418818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.419004 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.419120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.419232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] resource.caching = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.419345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.419457 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.419569 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.419690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.419804 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.419918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.420030 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.420142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.420254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.420378 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.420492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.420604 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.420729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.420841 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.420965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.421086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.421200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.421312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.421425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.421536 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] role.caching = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.421651 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.421771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] role.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.421891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] resource.caching = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.422005 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.422118 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.422230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.422342 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.422462 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.422576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.422688 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.422801 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.422912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.423025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.423146 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.423260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.423372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.423484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.423596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.423708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.423828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.423942 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.424054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.424168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] role.caching = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.424304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.424422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] role.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.424543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.424690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.424807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.424920 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.425032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.425148 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.425273 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.425447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.425564 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.425677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.425790 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.425935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.426061 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.426175 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.426288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.426460 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.426576 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.426689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.426809 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.426922 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.427034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.427175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.427294 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.427415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.427530 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.427666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.427783 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.427960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.428077 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.428198 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.428313 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.428426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.428529 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.428692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.428812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.428960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.429156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.429275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.429416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.429534 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.429646 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.429767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.429880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.429992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.430104 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] token.caching = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.430216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.430349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.430476 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.430589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.430701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.430813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.430925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.431036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.431156 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.431301 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.431419 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.431532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.431643 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.431787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.431916 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.432031 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.432143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.432911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.433118 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.433237 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.433363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.433477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.433591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] token.caching = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.433703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.433815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.433927 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.434048 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.434162 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.434275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.434387 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.434541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.434659 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.434785 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.434899 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.435011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.435124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.connection = **** {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.435245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.435366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.435480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.435600 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.435715 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.435828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.435970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.436087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.436200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.436320 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.436434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.436546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.436709 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.436828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.436941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.437064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.437179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.437292 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.437404 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.437517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.437651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.437777 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.437892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.438004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.connection = **** {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.438125 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.438247 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.438361 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.438473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.438660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.438777 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.438890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.439002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.439115 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.439227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.439347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.439461 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.439574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81793) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 12 11:59:49.439687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.439799 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.440013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.440253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.440502 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.440766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.441020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.441261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.441515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.441749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.441991 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.442229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 12 11:59:49.442399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81794) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 12 11:59:49.468440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.468700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.468934 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.470658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.470895 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.471124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.478664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.481346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.507986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.508222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.509764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.510001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.562174 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.562317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.612734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.612973 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.613530 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.613773 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.678559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.678794 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.679021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.679137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.679288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.679399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.679476 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.679592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.679673 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.679816 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.680038 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.680256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.703218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.703457 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.703543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.703692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.703781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.703914 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.704035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.704117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.704277 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.704370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.704504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.704584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.704756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.704978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.716031 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.716483 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.728690 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.728929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.729248 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.729348 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.729477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.729557 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.729720 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.729836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.729951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.730058 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.730175 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.730276 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81794) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.730394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.730613 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81793) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 12 11:59:49.747826 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.748038 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 12 11:59:49.748119 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.748254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 12 11:59:49.748388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.748594 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 12 11:59:49.748754 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.748829 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 12 11:59:49.748899 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.749092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.749228 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.749364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.749445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.749633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.749712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.749903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.761125 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.761407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.761638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.761722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.761935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.762015 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.762216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.762507 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.768887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.769556 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.771955 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.772248 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.772537 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.772660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.772854 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.772959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.773146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.773254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.773415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.773548 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.773694 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.773841 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.773973 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.774114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.774248 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.774397 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.774530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.774682 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.774806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.774961 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.775242 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.775516 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.798381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.799523 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.799883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.800175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.801041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.801335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.804987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.805283 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.805576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.805980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.806274 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.806597 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.812829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.813081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.813894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.814189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.817206 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.817508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.817801 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.818108 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.818304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.818441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.818608 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.818733 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.818911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.819035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.819206 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.819330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.819501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.819809 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.820096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.820374 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.835856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.836138 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.837083 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.837376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.840293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.841550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.843475 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.843778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.844729 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.845036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.848188 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.849458 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.851403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.852678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.854805 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.855120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.855414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.855715 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.856011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.856090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.856300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.856380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.856601 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.856927 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.857213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.857501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.873107 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.873409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.873689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.873965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.874122 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.874423 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.874701 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.874976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.881268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.882111 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.882859 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.883145 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.883700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.883988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.886640 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.886930 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.887238 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.887462 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.887546 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.887758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.887849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.888059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.888360 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.888660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.899246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.899539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.899832 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.900122 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.900205 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.900420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.900508 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.900732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.900816 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.901072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.901203 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.901368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.901659 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.901949 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.918851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.919181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.921844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.922188 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.924816 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.925165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.926395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.926752 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.927893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.928184 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.928263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.928487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.928588 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.928863 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.935318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.935778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.936977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.937517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.939925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.940227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.940460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.940539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.940788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.940929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.941090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.941374 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.949461 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.949761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.950053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.950390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.950692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.950986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.957231 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81794) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.957522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.957811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.957920 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81793) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 12 11:59:49.958093 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.958217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.958374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.958520 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.958655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.958814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.958935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.959101 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.959216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.959396 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.959602 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.959681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.959883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.959963 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.960162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.960326 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.960438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.960618 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.960747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.960929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.961010 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81794) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.961220 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.961509 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.961788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81793) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 12 11:59:49.995019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e09c4229c0 pid: 81794 (default app) Aug 12 11:59:49.996603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e09c4229c0 pid: 81793 (default app) Aug 12 11:59:49.998275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc1f22-4dac-4b4f-8e10-36f254528f01 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:59:49.998567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc1f22-4dac-4b4f-8e10-36f254528f01 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:59:49.998843 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc1f22-4dac-4b4f-8e10-36f254528f01 None None] PATH_INFO: `/v3` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:59:50.003146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Fri Aug 12 11:59:49 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 11:59:58.605551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da5744-4d2d-43de-a9a8-c1f667391ce1 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:59:58.606062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da5744-4d2d-43de-a9a8-c1f667391ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:59:58.606140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da5744-4d2d-43de-a9a8-c1f667391ce1 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:59:58.610570 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 11:59:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 11:59:58.615814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d6f9-3cdb-46df-9139-49657d62bcaa None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:59:58.616171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d6f9-3cdb-46df-9139-49657d62bcaa None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:59:58.616374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d6f9-3cdb-46df-9139-49657d62bcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:59:58.687559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: INFO dbcounter [None req-d959d6f9-3cdb-46df-9139-49657d62bcaa None None] Registered counter for database keystone Aug 12 11:59:58.706994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG oslo_db.sqlalchemy.engines [None req-d959d6f9-3cdb-46df-9139-49657d62bcaa None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81794) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 12 11:59:58.713204 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writer thread running {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 12 11:59:58.895548 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG passlib.handlers.bcrypt [None req-d959d6f9-3cdb-46df-9139-49657d62bcaa None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81794) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 12 11:59:58.895898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG passlib.handlers.bcrypt [None req-d959d6f9-3cdb-46df-9139-49657d62bcaa None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81794) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 12 11:59:58.950284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-d959d6f9-3cdb-46df-9139-49657d62bcaa None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 11:59:58.951889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-d959d6f9-3cdb-46df-9139-49657d62bcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 11:59:59.170758 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 11:59:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 556 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 12 11:59:59.633660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da5744-4d2d-43de-a9a8-c1f667391ce1 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:59:59.703281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: INFO dbcounter [None req-09da5744-4d2d-43de-a9a8-c1f667391ce1 None None] Registered counter for database keystone Aug 12 11:59:59.724314 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG oslo_db.sqlalchemy.engines [None req-09da5744-4d2d-43de-a9a8-c1f667391ce1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81793) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 12 11:59:59.731000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writer thread running {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 12 11:59:59.850112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da5744-4d2d-43de-a9a8-c1f667391ce1 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:59:59.854987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4d5c7-694b-4dbe-a7b8-d1357b7a1ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:59:59.855677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4d5c7-694b-4dbe-a7b8-d1357b7a1ade admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:59:59.855959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4d5c7-694b-4dbe-a7b8-d1357b7a1ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:59:59.856230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4d5c7-694b-4dbe-a7b8-d1357b7a1ade admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:59:59.864217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4d5c7-694b-4dbe-a7b8-d1357b7a1ade admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 11:59:59.915640 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: WARNING keystone.common.rbac_enforcer.enforcer [None req-49a4d5c7-694b-4dbe-a7b8-d1357b7a1ade admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 12 11:59:59.915991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4d5c7-694b-4dbe-a7b8-d1357b7a1ade admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 11:59:59.925919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Fri Aug 12 11:59:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 12 11:59:59.931848 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959d6f9-3cdb-46df-9139-49657d62bcaa None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 11:59:59.946942 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959d6f9-3cdb-46df-9139-49657d62bcaa None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 11:59:59.951463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99fac9-dc84-4eb8-8c36-26b5332b9f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 11:59:59.952548 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99fac9-dc84-4eb8-8c36-26b5332b9f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 11:59:59.952858 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99fac9-dc84-4eb8-8c36-26b5332b9f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 11:59:59.953131 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99fac9-dc84-4eb8-8c36-26b5332b9f91 admin admin] PATH_INFO: `/v3/projects/9e93c7bb262f46249ca80b95c794bf2f` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 11:59:59.954545 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99fac9-dc84-4eb8-8c36-26b5332b9f91 admin admin] RBAC: Authorizing `identity:get_project(project_id=9e93c7bb262f46249ca80b95c794bf2f)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:00.005435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0c99fac9-dc84-4eb8-8c36-26b5332b9f91 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 12 12:00:00.005781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99fac9-dc84-4eb8-8c36-26b5332b9f91 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:00.010309 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Fri Aug 12 11:59:59 2022] GET /identity/v3/projects/9e93c7bb262f46249ca80b95c794bf2f => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:01.892091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48011b0c-280d-4663-942f-759cafa76642 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:01.892621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48011b0c-280d-4663-942f-759cafa76642 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:01.892742 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48011b0c-280d-4663-942f-759cafa76642 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:01.896827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:00:01.902218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d2cd0-2b2c-41bd-99b7-a51bb76ff89a None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:01.902673 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d2cd0-2b2c-41bd-99b7-a51bb76ff89a None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:01.902765 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d2cd0-2b2c-41bd-99b7-a51bb76ff89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:01.924216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f84d2cd0-2b2c-41bd-99b7-a51bb76ff89a None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:01.925773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f84d2cd0-2b2c-41bd-99b7-a51bb76ff89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:01.933836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 12 12:00:02.200021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48011b0c-280d-4663-942f-759cafa76642 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:02.212117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48011b0c-280d-4663-942f-759cafa76642 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:02.217495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a546d4e-0e9b-40a1-9bab-f82cb2e7bc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:02.218091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a546d4e-0e9b-40a1-9bab-f82cb2e7bc21 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:02.218375 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a546d4e-0e9b-40a1-9bab-f82cb2e7bc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:02.218648 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a546d4e-0e9b-40a1-9bab-f82cb2e7bc21 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:02.226199 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a546d4e-0e9b-40a1-9bab-f82cb2e7bc21 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:02.227996 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a546d4e-0e9b-40a1-9bab-f82cb2e7bc21 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:02.236793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Fri Aug 12 12:00:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 12:00:02.242038 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d2cd0-2b2c-41bd-99b7-a51bb76ff89a None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:02.251728 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d2cd0-2b2c-41bd-99b7-a51bb76ff89a None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:02.256099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611d391-2cd4-43b9-9895-8ac35b7e6f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:02.256757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611d391-2cd4-43b9-9895-8ac35b7e6f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:02.257054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611d391-2cd4-43b9-9895-8ac35b7e6f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:02.257334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611d391-2cd4-43b9-9895-8ac35b7e6f6b admin admin] PATH_INFO: `/v3/users/1b637f155b514b58ab1c4d9c02381dc6` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:02.258649 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611d391-2cd4-43b9-9895-8ac35b7e6f6b admin admin] RBAC: Authorizing `identity:get_user(user_id=1b637f155b514b58ab1c4d9c02381dc6)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:02.261315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611d391-2cd4-43b9-9895-8ac35b7e6f6b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:02.262949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Fri Aug 12 12:00:02 2022] GET /identity/v3/users/1b637f155b514b58ab1c4d9c02381dc6 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:04.241633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca3af8-92d6-46c3-9237-a8ee0eb05979 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:04.242196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca3af8-92d6-46c3-9237-a8ee0eb05979 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:04.242196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca3af8-92d6-46c3-9237-a8ee0eb05979 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:04.243492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:04.248798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71980a-85fe-43de-a3b8-fba894ae6788 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:04.249203 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71980a-85fe-43de-a3b8-fba894ae6788 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:04.249490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71980a-85fe-43de-a3b8-fba894ae6788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:04.256984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91e2c8-a82d-4a65-b481-dcaf4a141327 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:04.257275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91e2c8-a82d-4a65-b481-dcaf4a141327 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:04.257553 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91e2c8-a82d-4a65-b481-dcaf4a141327 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:04.258824 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 6/11] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:00:04.264847 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de99fe-9569-4c7b-b357-ff0f5b57a095 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:04.265134 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de99fe-9569-4c7b-b357-ff0f5b57a095 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:04.265406 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de99fe-9569-4c7b-b357-ff0f5b57a095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:04.267904 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-cc71980a-85fe-43de-a3b8-fba894ae6788 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:04.269118 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-cc71980a-85fe-43de-a3b8-fba894ae6788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:04.307733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG passlib.handlers.bcrypt [None req-e1de99fe-9569-4c7b-b357-ff0f5b57a095 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81793) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 12 12:00:04.308150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG passlib.handlers.bcrypt [None req-e1de99fe-9569-4c7b-b357-ff0f5b57a095 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81793) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 12 12:00:04.348735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 6/12] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:04.354444 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-e1de99fe-9569-4c7b-b357-ff0f5b57a095 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:04.356080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-e1de99fe-9569-4c7b-b357-ff0f5b57a095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:04.452107 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:00:04.617754 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71980a-85fe-43de-a3b8-fba894ae6788 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:04.641215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:04.721740 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de99fe-9569-4c7b-b357-ff0f5b57a095 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:04.723596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71980a-85fe-43de-a3b8-fba894ae6788 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:04.757228 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa465ddf-9919-4a39-95dc-e180e553a5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:04.757858 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa465ddf-9919-4a39-95dc-e180e553a5b4 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:04.758137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa465ddf-9919-4a39-95dc-e180e553a5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:04.758414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa465ddf-9919-4a39-95dc-e180e553a5b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:04.770118 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa465ddf-9919-4a39-95dc-e180e553a5b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-aa465ddf-9919-4a39-95dc-e180e553a5b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:04.778490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:04.779533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:04.785216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:04.785216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:04.785216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:04.785216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:04.785216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:04.785216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:04.785216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:04.785216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:04.785216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:04.785216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:04.785216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:04.785216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:04.785216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 7/14] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:00:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:04.785849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa465ddf-9919-4a39-95dc-e180e553a5b4 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:04.825662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de99fe-9569-4c7b-b357-ff0f5b57a095 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:04.827964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:04.867065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f349c90-ef74-4f55-ae29-58bf6cf3b705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:04.868136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f349c90-ef74-4f55-ae29-58bf6cf3b705 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:04.868547 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f349c90-ef74-4f55-ae29-58bf6cf3b705 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:04.868938 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f349c90-ef74-4f55-ae29-58bf6cf3b705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:04.869046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:04.887358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa465ddf-9919-4a39-95dc-e180e553a5b4 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:04.922048 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8829a5-3675-4658-b4e3-4c7027f64aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:04.922652 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8829a5-3675-4658-b4e3-4c7027f64aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:04.922932 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8829a5-3675-4658-b4e3-4c7027f64aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:04.923206 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8829a5-3675-4658-b4e3-4c7027f64aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:04.923875 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8829a5-3675-4658-b4e3-4c7027f64aa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:04.924177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8829a5-3675-4658-b4e3-4c7027f64aa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:04.925611 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8829a5-3675-4658-b4e3-4c7027f64aa6 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:04.937498 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 8/15] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:04.937840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f349c90-ef74-4f55-ae29-58bf6cf3b705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:04.939323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f349c90-ef74-4f55-ae29-58bf6cf3b705 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:04.943024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8829a5-3675-4658-b4e3-4c7027f64aa6 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:05.041918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 8/16] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 12 12:00:05.043240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8829a5-3675-4658-b4e3-4c7027f64aa6 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:05.047466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f349c90-ef74-4f55-ae29-58bf6cf3b705 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:05.050211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:05.152514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f349c90-ef74-4f55-ae29-58bf6cf3b705 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:05.184332 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c47478-6cac-4de3-bb69-4a87f886890e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:05.184976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c47478-6cac-4de3-bb69-4a87f886890e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:05.185263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c47478-6cac-4de3-bb69-4a87f886890e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:05.185537 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c47478-6cac-4de3-bb69-4a87f886890e None admin] PATH_INFO: `/v3/users/1b637f155b514b58ab1c4d9c02381dc6` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:05.186215 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682b1d0-a7a1-41fa-b071-f176cf5a381b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:05.186685 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c47478-6cac-4de3-bb69-4a87f886890e None admin] RBAC: Authorizing `identity:get_user(user_id=1b637f155b514b58ab1c4d9c02381dc6)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:05.187052 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682b1d0-a7a1-41fa-b071-f176cf5a381b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:05.187332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682b1d0-a7a1-41fa-b071-f176cf5a381b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:05.187599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682b1d0-a7a1-41fa-b071-f176cf5a381b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:05.188141 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c47478-6cac-4de3-bb69-4a87f886890e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:05.189738 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 9/17] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:04 2022] GET /identity/v3/users/1b637f155b514b58ab1c4d9c02381dc6 => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:05.195154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c47478-6cac-4de3-bb69-4a87f886890e None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:05.206644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682b1d0-a7a1-41fa-b071-f176cf5a381b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:05.207906 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682b1d0-a7a1-41fa-b071-f176cf5a381b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:05.210534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-4682b1d0-a7a1-41fa-b071-f176cf5a381b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:00:05.226246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:05.227133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:00:05.227911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:00:05.227911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:00:05.227911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:05.227911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 9/18] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:00:05.233377 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682b1d0-a7a1-41fa-b071-f176cf5a381b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:05.292179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:05.297700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c47478-6cac-4de3-bb69-4a87f886890e None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:05.331539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797246f-3301-4bd1-8cbc-6bf473fc424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:05.332167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797246f-3301-4bd1-8cbc-6bf473fc424d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:05.332453 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797246f-3301-4bd1-8cbc-6bf473fc424d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:05.332767 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797246f-3301-4bd1-8cbc-6bf473fc424d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:05.333900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797246f-3301-4bd1-8cbc-6bf473fc424d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:05.335343 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797246f-3301-4bd1-8cbc-6bf473fc424d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:05.335703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682b1d0-a7a1-41fa-b071-f176cf5a381b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:05.336943 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 10/19] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:05.343250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797246f-3301-4bd1-8cbc-6bf473fc424d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:05.370275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33689e-d6f1-4998-9f6a-791839e1062a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:05.370877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33689e-d6f1-4998-9f6a-791839e1062a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:05.371154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33689e-d6f1-4998-9f6a-791839e1062a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:05.371429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33689e-d6f1-4998-9f6a-791839e1062a None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:05.372063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33689e-d6f1-4998-9f6a-791839e1062a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:05.372361 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33689e-d6f1-4998-9f6a-791839e1062a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:05.373778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33689e-d6f1-4998-9f6a-791839e1062a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:05.391485 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 10/20] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:05.425005 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:05.446039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797246f-3301-4bd1-8cbc-6bf473fc424d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:05.483436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831e810-36b6-4af7-8350-6eff4a9944ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:05.484057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831e810-36b6-4af7-8350-6eff4a9944ca None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:05.484338 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831e810-36b6-4af7-8350-6eff4a9944ca None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:05.484609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831e810-36b6-4af7-8350-6eff4a9944ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:05.485380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831e810-36b6-4af7-8350-6eff4a9944ca None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.domain.id=default, user.id=1b637f155b514b58ab1c4d9c02381dc6) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:05.485685 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831e810-36b6-4af7-8350-6eff4a9944ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:05.487149 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831e810-36b6-4af7-8350-6eff4a9944ca None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:05.494586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 11/21] 10.222.0.27 () {56 vars in 1322 bytes} [Fri Aug 12 12:00:05 2022] GET /identity/v3/role_assignments?user.id=1b637f155b514b58ab1c4d9c02381dc6&scope.domain.id=default&role.id=906e3a8a6c7540a38f629f2567133347 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:07.546908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8bf75-aaae-456a-9a45-68b0836cb7b9 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:07.547942 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8bf75-aaae-456a-9a45-68b0836cb7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:07.548426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8bf75-aaae-456a-9a45-68b0836cb7b9 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:07.550373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 11/22] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:07.555902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a324fd3-1544-49b7-8616-f3a51d7c1a22 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:07.557543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a324fd3-1544-49b7-8616-f3a51d7c1a22 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:07.558021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a324fd3-1544-49b7-8616-f3a51d7c1a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:07.576808 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-5a324fd3-1544-49b7-8616-f3a51d7c1a22 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:07.578335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-5a324fd3-1544-49b7-8616-f3a51d7c1a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:07.673013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 12/23] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:07.725943 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333198a0-34d4-42c3-85f4-3369ea394465 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:07.726574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333198a0-34d4-42c3-85f4-3369ea394465 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:07.727049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333198a0-34d4-42c3-85f4-3369ea394465 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:07.728980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 12/24] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:07.734532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1a019-b5e4-433d-bce6-a5a3cef492f7 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:07.735047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1a019-b5e4-433d-bce6-a5a3cef492f7 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:07.735517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1a019-b5e4-433d-bce6-a5a3cef492f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:07.754717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9de1a019-b5e4-433d-bce6-a5a3cef492f7 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:07.756261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9de1a019-b5e4-433d-bce6-a5a3cef492f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:07.839367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 13/25] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:08.024835 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc8565-f227-49f7-8392-666f39ed162c None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:08.025582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc8565-f227-49f7-8392-666f39ed162c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:08.026062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc8565-f227-49f7-8392-666f39ed162c None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:08.027963 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 13/26] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:08.033524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fd386-30f4-468b-80cc-d32f335e5347 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:08.034064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fd386-30f4-468b-80cc-d32f335e5347 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:08.034535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fd386-30f4-468b-80cc-d32f335e5347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:08.042794 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:08.058192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-550fd386-30f4-468b-80cc-d32f335e5347 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:08.059735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-550fd386-30f4-468b-80cc-d32f335e5347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:08.142416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 14/27] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:08.167494 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc8565-f227-49f7-8392-666f39ed162c None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:08.302282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc8565-f227-49f7-8392-666f39ed162c None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:08.317403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666ef6d-346e-49a4-8475-e88052867d62 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:08.318031 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666ef6d-346e-49a4-8475-e88052867d62 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:08.318565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666ef6d-346e-49a4-8475-e88052867d62 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:08.320090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 15/28] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:00:08.325026 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:08.329608 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd57d5-35fc-4256-8310-0a24d187f6fc None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:08.329903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd57d5-35fc-4256-8310-0a24d187f6fc None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:08.330188 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd57d5-35fc-4256-8310-0a24d187f6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:08.359839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-6ccd57d5-35fc-4256-8310-0a24d187f6fc None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:08.361386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-6ccd57d5-35fc-4256-8310-0a24d187f6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:08.370870 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff12686-3b55-406c-b3a0-b5a7ba82fa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:08.371516 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff12686-3b55-406c-b3a0-b5a7ba82fa0e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:08.371802 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff12686-3b55-406c-b3a0-b5a7ba82fa0e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:08.372079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff12686-3b55-406c-b3a0-b5a7ba82fa0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:08.392353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff12686-3b55-406c-b3a0-b5a7ba82fa0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:08.393864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff12686-3b55-406c-b3a0-b5a7ba82fa0e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-9ff12686-3b55-406c-b3a0-b5a7ba82fa0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:00:08.418059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:08.418978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:00:08.424828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:00:08.424828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:00:08.424828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:08.424828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 14/29] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:08.424828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff12686-3b55-406c-b3a0-b5a7ba82fa0e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:08.448189 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 16/30] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:00:08.449731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd57d5-35fc-4256-8310-0a24d187f6fc None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:08.543800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:08.577715 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff12686-3b55-406c-b3a0-b5a7ba82fa0e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:08.610323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd57d5-35fc-4256-8310-0a24d187f6fc None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:08.678556 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c68331-6a5e-44e3-9a0d-da674eabf81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:08.679203 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c68331-6a5e-44e3-9a0d-da674eabf81f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:08.679492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c68331-6a5e-44e3-9a0d-da674eabf81f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:08.679773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c68331-6a5e-44e3-9a0d-da674eabf81f None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:08.680451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c68331-6a5e-44e3-9a0d-da674eabf81f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:08.680786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c68331-6a5e-44e3-9a0d-da674eabf81f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:08.682631 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c68331-6a5e-44e3-9a0d-da674eabf81f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:08.716791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 17/31] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:08.716791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c68331-6a5e-44e3-9a0d-da674eabf81f None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:08.736930 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93766b75-2c01-4252-be3c-bfda9c589dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:08.740477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93766b75-2c01-4252-be3c-bfda9c589dbd None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:08.742124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93766b75-2c01-4252-be3c-bfda9c589dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:08.742594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93766b75-2c01-4252-be3c-bfda9c589dbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:08.765179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93766b75-2c01-4252-be3c-bfda9c589dbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-93766b75-2c01-4252-be3c-bfda9c589dbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:08.775096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:08.776260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:08.777029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:08.777029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:08.777029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:08.777029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:08.777029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:08.777029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:08.777029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:08.777029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:08.777029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:08.777029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:08.777029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:08.777029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:08.782936 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 15/32] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:00:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:08.785990 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93766b75-2c01-4252-be3c-bfda9c589dbd None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:08.849496 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:08.864587 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c68331-6a5e-44e3-9a0d-da674eabf81f None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:08.892409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:08.905220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996ffab-0cd9-4e37-918b-20384fc56daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:08.905834 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996ffab-0cd9-4e37-918b-20384fc56daf None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:08.906119 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996ffab-0cd9-4e37-918b-20384fc56daf None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:08.906397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996ffab-0cd9-4e37-918b-20384fc56daf None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:08.907067 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996ffab-0cd9-4e37-918b-20384fc56daf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:08.908469 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996ffab-0cd9-4e37-918b-20384fc56daf None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:08.938330 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93766b75-2c01-4252-be3c-bfda9c589dbd None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:08.974173 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c48e5b-7847-45b7-b233-d20d521bcc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:08.974782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c48e5b-7847-45b7-b233-d20d521bcc61 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:08.975060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c48e5b-7847-45b7-b233-d20d521bcc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:08.975329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c48e5b-7847-45b7-b233-d20d521bcc61 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:08.976174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c48e5b-7847-45b7-b233-d20d521bcc61 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:08.977650 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c48e5b-7847-45b7-b233-d20d521bcc61 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:09.110485 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 18/33] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Aug 12 12:00:08 2022] POST /identity/v3/roles => generated 219 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:00:09.111402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996ffab-0cd9-4e37-918b-20384fc56daf None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:09.201270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 16/34] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 12 12:00:08 2022] POST /identity/v3/projects => generated 301 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:00:09.202412 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c48e5b-7847-45b7-b233-d20d521bcc61 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:09.240666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996ffab-0cd9-4e37-918b-20384fc56daf None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:09.289432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be91f91-1b67-4f53-beee-4d75d85ec109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:09.293064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be91f91-1b67-4f53-beee-4d75d85ec109 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:09.293637 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be91f91-1b67-4f53-beee-4d75d85ec109 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:09.294129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be91f91-1b67-4f53-beee-4d75d85ec109 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:09.295663 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be91f91-1b67-4f53-beee-4d75d85ec109 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:09.296218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be91f91-1b67-4f53-beee-4d75d85ec109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:09.299659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be91f91-1b67-4f53-beee-4d75d85ec109 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:09.301807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:09.312494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 19/35] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:09.316687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ca91d-1d7b-4a07-a4a6-4c2219b072e4 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:09.316687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ca91d-1d7b-4a07-a4a6-4c2219b072e4 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:09.316687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ca91d-1d7b-4a07-a4a6-4c2219b072e4 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:09.316687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 20/36] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:09.320774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ca91d-1d7b-4a07-a4a6-4c2219b072e4 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:09.334096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c48e5b-7847-45b7-b233-d20d521bcc61 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:09.378878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aecd65-804c-43a1-a9cf-6edff83e7a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:09.379517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aecd65-804c-43a1-a9cf-6edff83e7a30 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:09.379872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aecd65-804c-43a1-a9cf-6edff83e7a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:09.380218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aecd65-804c-43a1-a9cf-6edff83e7a30 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:09.381013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aecd65-804c-43a1-a9cf-6edff83e7a30 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:09.382919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aecd65-804c-43a1-a9cf-6edff83e7a30 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:09.391475 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:09.455466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ca91d-1d7b-4a07-a4a6-4c2219b072e4 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:09.455928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 17/37] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Aug 12 12:00:09 2022] POST /identity/v3/roles => generated 225 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:00:09.457779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba06728-9656-4a6e-ba3a-c104dee68393 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:09.458262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba06728-9656-4a6e-ba3a-c104dee68393 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:09.459460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba06728-9656-4a6e-ba3a-c104dee68393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:09.487085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-0ba06728-9656-4a6e-ba3a-c104dee68393 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:09.488771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0ba06728-9656-4a6e-ba3a-c104dee68393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:09.502238 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69db353-f873-40c7-b5bd-c34f2df80096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:09.503325 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69db353-f873-40c7-b5bd-c34f2df80096 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:09.503891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69db353-f873-40c7-b5bd-c34f2df80096 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:09.504480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69db353-f873-40c7-b5bd-c34f2df80096 None admin] PATH_INFO: `/v3/users/1b637f155b514b58ab1c4d9c02381dc6` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:09.506124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69db353-f873-40c7-b5bd-c34f2df80096 None admin] RBAC: Authorizing `identity:get_user(user_id=1b637f155b514b58ab1c4d9c02381dc6)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:09.508150 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69db353-f873-40c7-b5bd-c34f2df80096 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:09.510775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 21/38] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:09 2022] GET /identity/v3/users/1b637f155b514b58ab1c4d9c02381dc6 => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:09.519752 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69db353-f873-40c7-b5bd-c34f2df80096 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:09.567487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 18/39] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:09.639999 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69db353-f873-40c7-b5bd-c34f2df80096 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:09.654852 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:09.667198 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf568e61-fa9c-4af3-89d0-53d1e42afc0f None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:09.679402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf568e61-fa9c-4af3-89d0-53d1e42afc0f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:09.679402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf568e61-fa9c-4af3-89d0-53d1e42afc0f None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:09.679402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 19/40] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:00:09.679402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fc86e-ee02-4b52-8e8d-86c7781f6509 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:09.679402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fc86e-ee02-4b52-8e8d-86c7781f6509 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:09.679402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fc86e-ee02-4b52-8e8d-86c7781f6509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:09.679769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2eae5-a09a-448b-b191-6766f728a96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:09.679769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2eae5-a09a-448b-b191-6766f728a96d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:09.679769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2eae5-a09a-448b-b191-6766f728a96d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:09.679769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2eae5-a09a-448b-b191-6766f728a96d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:09.680002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2eae5-a09a-448b-b191-6766f728a96d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:09.680422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2eae5-a09a-448b-b191-6766f728a96d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:09.682473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 22/41] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:09.690449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2eae5-a09a-448b-b191-6766f728a96d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:09.698308 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-178fc86e-ee02-4b52-8e8d-86c7781f6509 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:09.701027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-178fc86e-ee02-4b52-8e8d-86c7781f6509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:09.778071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:09.783837 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 20/42] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:00:09.810169 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2eae5-a09a-448b-b191-6766f728a96d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:09.847751 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b50fb-ea36-4619-953a-8f124985c0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:09.848792 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b50fb-ea36-4619-953a-8f124985c0fa None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:09.849082 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b50fb-ea36-4619-953a-8f124985c0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:09.849318 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fc86e-ee02-4b52-8e8d-86c7781f6509 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:09.849418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b50fb-ea36-4619-953a-8f124985c0fa None admin] PATH_INFO: `/v3/domains/default/users/1b637f155b514b58ab1c4d9c02381dc6/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:09.851346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b50fb-ea36-4619-953a-8f124985c0fa None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1b637f155b514b58ab1c4d9c02381dc6, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:09.854061 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b50fb-ea36-4619-953a-8f124985c0fa None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:09.948424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fc86e-ee02-4b52-8e8d-86c7781f6509 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:09.954500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 23/43] 10.222.0.27 () {58 vars in 1361 bytes} [Fri Aug 12 12:00:09 2022] PUT /identity/v3/domains/default/users/1b637f155b514b58ab1c4d9c02381dc6/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:00:09.997633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29117a05-ab20-4f83-beff-bff55104f3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:09.998616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29117a05-ab20-4f83-beff-bff55104f3dd None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:09.998905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29117a05-ab20-4f83-beff-bff55104f3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:09.999190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29117a05-ab20-4f83-beff-bff55104f3dd None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:09.999886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29117a05-ab20-4f83-beff-bff55104f3dd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:10.001416 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29117a05-ab20-4f83-beff-bff55104f3dd None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:10.056279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b50fb-ea36-4619-953a-8f124985c0fa None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:10.086515 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 21/44] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Aug 12 12:00:09 2022] POST /identity/v3/roles => generated 223 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:00:10.109422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:10.175500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b50fb-ea36-4619-953a-8f124985c0fa None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:10.212295 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66889bd3-a300-491c-b4d7-4389ad8ac355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:10.213474 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66889bd3-a300-491c-b4d7-4389ad8ac355 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:10.213755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66889bd3-a300-491c-b4d7-4389ad8ac355 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:10.214021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66889bd3-a300-491c-b4d7-4389ad8ac355 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:10.215226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66889bd3-a300-491c-b4d7-4389ad8ac355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:10.216750 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66889bd3-a300-491c-b4d7-4389ad8ac355 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:10.218578 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 24/45] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:10.223989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29117a05-ab20-4f83-beff-bff55104f3dd None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:10.286705 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:10.329235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29117a05-ab20-4f83-beff-bff55104f3dd None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:10.362804 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774ee34-70ee-422e-bf55-bc6c85017366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:10.363585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774ee34-70ee-422e-bf55-bc6c85017366 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:10.364043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774ee34-70ee-422e-bf55-bc6c85017366 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:10.364850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774ee34-70ee-422e-bf55-bc6c85017366 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:10.365518 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774ee34-70ee-422e-bf55-bc6c85017366 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:10.366978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774ee34-70ee-422e-bf55-bc6c85017366 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:10.451162 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 22/46] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 12 12:00:10 2022] POST /identity/v3/projects => generated 312 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:00:11.885656 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821e4f9-6372-4e8c-8e1e-e4fbc484cd58 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:11.886165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821e4f9-6372-4e8c-8e1e-e4fbc484cd58 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:11.886267 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821e4f9-6372-4e8c-8e1e-e4fbc484cd58 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:11.887536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 25/47] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:11.892629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6f47b-a02f-4a63-ab3e-4870aefbc34c None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:11.893031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6f47b-a02f-4a63-ab3e-4870aefbc34c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:11.893320 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6f47b-a02f-4a63-ab3e-4870aefbc34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:11.911747 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-bcd6f47b-a02f-4a63-ab3e-4870aefbc34c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:11.912954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-bcd6f47b-a02f-4a63-ab3e-4870aefbc34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:11.979689 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 23/48] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:12.249470 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821e4f9-6372-4e8c-8e1e-e4fbc484cd58 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:12.349319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8d5d1-0455-46e2-baed-379d319724b4 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:12.349611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8d5d1-0455-46e2-baed-379d319724b4 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:12.349704 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821e4f9-6372-4e8c-8e1e-e4fbc484cd58 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:12.349911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8d5d1-0455-46e2-baed-379d319724b4 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:12.351260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 24/49] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:00:12.356181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c60d47-d4a6-4812-a99f-200d6b054ed8 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:12.356466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c60d47-d4a6-4812-a99f-200d6b054ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:12.356779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c60d47-d4a6-4812-a99f-200d6b054ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:12.360485 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:12.374629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-33c60d47-d4a6-4812-a99f-200d6b054ed8 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:12.375825 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-33c60d47-d4a6-4812-a99f-200d6b054ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:12.385640 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85326005-a897-4d91-bc7d-c1f1fab4125a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:12.386353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85326005-a897-4d91-bc7d-c1f1fab4125a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:12.386644 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85326005-a897-4d91-bc7d-c1f1fab4125a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:12.386929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85326005-a897-4d91-bc7d-c1f1fab4125a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:12.398108 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85326005-a897-4d91-bc7d-c1f1fab4125a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-85326005-a897-4d91-bc7d-c1f1fab4125a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:12.403320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:12.404263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:12.405037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:12.405037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:12.405037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:12.405037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:12.405037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:12.405037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:12.405037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:12.405037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:12.405037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:12.405037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:12.405037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:12.405037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:12.409295 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 26/50] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:00:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:12.410244 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85326005-a897-4d91-bc7d-c1f1fab4125a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:12.411924 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:12.444973 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 25/51] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:00:12.508345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85326005-a897-4d91-bc7d-c1f1fab4125a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:12.540329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bede84-6c32-4710-ab23-75f92287964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:12.541483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bede84-6c32-4710-ab23-75f92287964d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:12.541788 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bede84-6c32-4710-ab23-75f92287964d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:12.542044 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bede84-6c32-4710-ab23-75f92287964d None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:12.542708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bede84-6c32-4710-ab23-75f92287964d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:12.543014 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bede84-6c32-4710-ab23-75f92287964d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:12.544420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bede84-6c32-4710-ab23-75f92287964d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:12.552183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 27/52] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:12.558603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c60d47-d4a6-4812-a99f-200d6b054ed8 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:12.653449 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c60d47-d4a6-4812-a99f-200d6b054ed8 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:12.685685 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6aa8f-de60-477e-a1c8-aa021e61f3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:12.686305 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6aa8f-de60-477e-a1c8-aa021e61f3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:12.686593 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6aa8f-de60-477e-a1c8-aa021e61f3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:12.686873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6aa8f-de60-477e-a1c8-aa021e61f3e4 None admin] PATH_INFO: `/v3/users/1b637f155b514b58ab1c4d9c02381dc6` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:12.688220 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6aa8f-de60-477e-a1c8-aa021e61f3e4 None admin] RBAC: Authorizing `identity:get_user(user_id=1b637f155b514b58ab1c4d9c02381dc6)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:12.689693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6aa8f-de60-477e-a1c8-aa021e61f3e4 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:12.691253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 26/53] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:12 2022] GET /identity/v3/users/1b637f155b514b58ab1c4d9c02381dc6 => generated 247 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:12.696835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bede84-6c32-4710-ab23-75f92287964d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:12.716959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6aa8f-de60-477e-a1c8-aa021e61f3e4 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:12.730429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:12.759214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:12.803781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bede84-6c32-4710-ab23-75f92287964d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:12.819888 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6aa8f-de60-477e-a1c8-aa021e61f3e4 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:12.840849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a73e30-45e0-4bc4-aed2-7a1c36215505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:12.841471 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a73e30-45e0-4bc4-aed2-7a1c36215505 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:12.841752 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a73e30-45e0-4bc4-aed2-7a1c36215505 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:12.842022 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a73e30-45e0-4bc4-aed2-7a1c36215505 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:12.843121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a73e30-45e0-4bc4-aed2-7a1c36215505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:12.844574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a73e30-45e0-4bc4-aed2-7a1c36215505 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:12.846172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 28/54] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:12.851959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a73e30-45e0-4bc4-aed2-7a1c36215505 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:12.853080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b05694-9b3a-4fd3-bc38-e936aa4bfeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:12.854298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b05694-9b3a-4fd3-bc38-e936aa4bfeb4 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:12.854579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b05694-9b3a-4fd3-bc38-e936aa4bfeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:12.854855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b05694-9b3a-4fd3-bc38-e936aa4bfeb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:12.855975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b05694-9b3a-4fd3-bc38-e936aa4bfeb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:12.857439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b05694-9b3a-4fd3-bc38-e936aa4bfeb4 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:12.858991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 27/55] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:12.864308 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b05694-9b3a-4fd3-bc38-e936aa4bfeb4 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:12.946157 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:12.949666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a73e30-45e0-4bc4-aed2-7a1c36215505 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:12.960489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b05694-9b3a-4fd3-bc38-e936aa4bfeb4 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:12.970882 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:12.984748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f197fc-9ba3-4b70-a386-a48ae16fa75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:12.985356 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f197fc-9ba3-4b70-a386-a48ae16fa75e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:12.985639 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f197fc-9ba3-4b70-a386-a48ae16fa75e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:12.985915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f197fc-9ba3-4b70-a386-a48ae16fa75e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:12.986636 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f197fc-9ba3-4b70-a386-a48ae16fa75e None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.domain.id=default, user.id=1b637f155b514b58ab1c4d9c02381dc6) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:12.986943 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f197fc-9ba3-4b70-a386-a48ae16fa75e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:12.988399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f197fc-9ba3-4b70-a386-a48ae16fa75e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:12.994098 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 29/56] 10.222.0.27 () {56 vars in 1322 bytes} [Fri Aug 12 12:00:12 2022] GET /identity/v3/role_assignments?user.id=1b637f155b514b58ab1c4d9c02381dc6&scope.domain.id=default&role.id=906e3a8a6c7540a38f629f2567133347 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:12.996314 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10aaf7-8ff4-4210-97d3-36cf9a0d458c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:13.000897 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10aaf7-8ff4-4210-97d3-36cf9a0d458c None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:13.000897 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10aaf7-8ff4-4210-97d3-36cf9a0d458c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:13.000897 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10aaf7-8ff4-4210-97d3-36cf9a0d458c None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:13.000897 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10aaf7-8ff4-4210-97d3-36cf9a0d458c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:13.000897 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10aaf7-8ff4-4210-97d3-36cf9a0d458c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:13.156007 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 28/57] 10.222.0.27 () {60 vars in 1140 bytes} [Fri Aug 12 12:00:12 2022] POST /identity/v3/projects => generated 298 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:00:15.041936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e107a-819f-4f2c-8e62-9a31f7aa0ef6 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:15.042444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e107a-819f-4f2c-8e62-9a31f7aa0ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:15.042555 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e107a-819f-4f2c-8e62-9a31f7aa0ef6 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:15.043833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 30/58] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:15.049010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb164fa-05e6-44d2-b609-ccfec63bc3b2 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:15.049397 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb164fa-05e6-44d2-b609-ccfec63bc3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:15.049567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb164fa-05e6-44d2-b609-ccfec63bc3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:15.067315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-6fb164fa-05e6-44d2-b609-ccfec63bc3b2 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:15.068431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6fb164fa-05e6-44d2-b609-ccfec63bc3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:15.132444 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 29/59] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:15.402277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e107a-819f-4f2c-8e62-9a31f7aa0ef6 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:15.505306 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e107a-819f-4f2c-8e62-9a31f7aa0ef6 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:15.537351 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6068965-e0b0-432b-8ee5-22aa1baea71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:15.537966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6068965-e0b0-432b-8ee5-22aa1baea71f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:15.538250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6068965-e0b0-432b-8ee5-22aa1baea71f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:15.538526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6068965-e0b0-432b-8ee5-22aa1baea71f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:15.539633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6068965-e0b0-432b-8ee5-22aa1baea71f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:15.541700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6068965-e0b0-432b-8ee5-22aa1baea71f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:15.543180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 31/60] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:15.548656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb164fa-05e6-44d2-b609-ccfec63bc3b2 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:15.637020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:15.646267 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb164fa-05e6-44d2-b609-ccfec63bc3b2 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:15.679444 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c7a3e-f4f8-4b4b-a450-1e986aa54e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:15.679983 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c7a3e-f4f8-4b4b-a450-1e986aa54e2a None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:15.680270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c7a3e-f4f8-4b4b-a450-1e986aa54e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:15.680547 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c7a3e-f4f8-4b4b-a450-1e986aa54e2a None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:15.681515 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c7a3e-f4f8-4b4b-a450-1e986aa54e2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:15.683420 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c7a3e-f4f8-4b4b-a450-1e986aa54e2a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:15.758988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 30/61] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:00:15 2022] POST /identity/v3/users => generated 275 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:00:18.833918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17962f16-9adb-4c71-8e41-bdc58c28362f None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:18.834439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17962f16-9adb-4c71-8e41-bdc58c28362f None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:18.834512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17962f16-9adb-4c71-8e41-bdc58c28362f None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:18.835810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 32/62] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:18.841574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65d8fe-4652-43b9-8164-8df56fc744be None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:18.841980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65d8fe-4652-43b9-8164-8df56fc744be None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:18.842329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65d8fe-4652-43b9-8164-8df56fc744be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:18.864431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-2d65d8fe-4652-43b9-8164-8df56fc744be None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:18.865822 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2d65d8fe-4652-43b9-8164-8df56fc744be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:18.926154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccbbdb-f9f0-4335-9c2e-a61547c458ba None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:18.926471 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccbbdb-f9f0-4335-9c2e-a61547c458ba None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:18.926718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccbbdb-f9f0-4335-9c2e-a61547c458ba None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:18.928061 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 33/63] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:18.944024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb206ba-1c69-49e6-856b-a22a141abcb9 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:18.944305 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb206ba-1c69-49e6-856b-a22a141abcb9 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:18.944587 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb206ba-1c69-49e6-856b-a22a141abcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:18.969602 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 31/64] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:18.970278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-0eb206ba-1c69-49e6-856b-a22a141abcb9 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:18.971815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0eb206ba-1c69-49e6-856b-a22a141abcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:18.999242 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e46af-1699-4734-936d-6369f6d540de None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:18.999534 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e46af-1699-4734-936d-6369f6d540de None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:18.999971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e46af-1699-4734-936d-6369f6d540de None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:19.001536 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 32/65] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:00:19.006889 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d014ae7-6346-4b8c-8518-f7d190575814 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:19.007347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d014ae7-6346-4b8c-8518-f7d190575814 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:19.007769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d014ae7-6346-4b8c-8518-f7d190575814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:19.025518 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:19.027126 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-8d014ae7-6346-4b8c-8518-f7d190575814 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:19.028460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8d014ae7-6346-4b8c-8518-f7d190575814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:19.104824 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 33/66] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:19.130927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 34/67] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 12 12:00:19.254492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54942ed4-888e-4d72-a868-1d21b7e81283 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:19.255124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54942ed4-888e-4d72-a868-1d21b7e81283 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:19.255774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54942ed4-888e-4d72-a868-1d21b7e81283 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:19.257656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 34/68] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:19.263051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8d674-bd4c-4c2d-a5d1-55ac3226588d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:19.263346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8d674-bd4c-4c2d-a5d1-55ac3226588d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:19.263631 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8d674-bd4c-4c2d-a5d1-55ac3226588d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:19.301625 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-1db8d674-bd4c-4c2d-a5d1-55ac3226588d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:19.302841 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-1db8d674-bd4c-4c2d-a5d1-55ac3226588d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:19.382485 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54942ed4-888e-4d72-a868-1d21b7e81283 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:19.391502 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 35/69] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:19.393407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8d674-bd4c-4c2d-a5d1-55ac3226588d None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:19.397111 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:19.497085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54942ed4-888e-4d72-a868-1d21b7e81283 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:19.540617 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:19.542128 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf6aef-2f8f-43fb-b03e-c2f320e78f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:19.543263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf6aef-2f8f-43fb-b03e-c2f320e78f6a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:19.543927 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf6aef-2f8f-43fb-b03e-c2f320e78f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:19.544749 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf6aef-2f8f-43fb-b03e-c2f320e78f6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:19.550173 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8d674-bd4c-4c2d-a5d1-55ac3226588d None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:19.559241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf6aef-2f8f-43fb-b03e-c2f320e78f6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-6cdf6aef-2f8f-43fb-b03e-c2f320e78f6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:19.565948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:19.566785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:19.567533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:19.567533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:19.567533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:19.567533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:19.567533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:19.567533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:19.567533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:19.567533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:19.567533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:19.567533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:19.567533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:19.567533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:19.569714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 35/70] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:00:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:19.574978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf6aef-2f8f-43fb-b03e-c2f320e78f6a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:19.599529 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ddc9a-3fed-457e-930b-b905693ba64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:19.601160 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ddc9a-3fed-457e-930b-b905693ba64a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:19.601951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ddc9a-3fed-457e-930b-b905693ba64a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:19.602432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ddc9a-3fed-457e-930b-b905693ba64a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:19.617467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ddc9a-3fed-457e-930b-b905693ba64a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-a78ddc9a-3fed-457e-930b-b905693ba64a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:19.624258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:19.625123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:19.630836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:19.630836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:19.630836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:19.630836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:19.630836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:19.630836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:19.630836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:19.630836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:19.630836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:19.630836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:19.630836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:19.630836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:19.630836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 36/71] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:19.630836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ddc9a-3fed-457e-930b-b905693ba64a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:19.646418 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:19.684476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf6aef-2f8f-43fb-b03e-c2f320e78f6a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:19.721321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c21d3-2964-418a-8f89-fa6344c99537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:19.721944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c21d3-2964-418a-8f89-fa6344c99537 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:19.722228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c21d3-2964-418a-8f89-fa6344c99537 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:19.722506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c21d3-2964-418a-8f89-fa6344c99537 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:19.729125 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ddc9a-3fed-457e-930b-b905693ba64a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:19.732788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c21d3-2964-418a-8f89-fa6344c99537 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-8f6c21d3-2964-418a-8f89-fa6344c99537 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:19.737524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:19.738533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:19.739506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:19.739506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:19.739506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:19.739506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:19.739506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:19.739506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:19.739506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:19.739506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:19.739506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:19.739506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:19.739506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:19.739506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:19.739506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 36/72] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:00:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:19.740216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3925ae-1b6d-407c-8003-199b409d60e5 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:19.740216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3925ae-1b6d-407c-8003-199b409d60e5 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:19.740327 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3925ae-1b6d-407c-8003-199b409d60e5 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:19.741524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 37/73] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:19.742216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3925ae-1b6d-407c-8003-199b409d60e5 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:19.767459 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523d9ca-eaa4-49da-b77b-d54f7fb90b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:19.768060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523d9ca-eaa4-49da-b77b-d54f7fb90b78 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:19.768344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523d9ca-eaa4-49da-b77b-d54f7fb90b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:19.768622 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523d9ca-eaa4-49da-b77b-d54f7fb90b78 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:19.769312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523d9ca-eaa4-49da-b77b-d54f7fb90b78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:19.769615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523d9ca-eaa4-49da-b77b-d54f7fb90b78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:19.770993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523d9ca-eaa4-49da-b77b-d54f7fb90b78 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:19.779560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 37/74] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:19.780386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523d9ca-eaa4-49da-b77b-d54f7fb90b78 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:19.800430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:19.849327 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3925ae-1b6d-407c-8003-199b409d60e5 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:19.873882 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:19.882060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf00a42-4a70-46b0-a927-392d61f009f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:19.882669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf00a42-4a70-46b0-a927-392d61f009f5 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:19.882951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf00a42-4a70-46b0-a927-392d61f009f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:19.883224 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf00a42-4a70-46b0-a927-392d61f009f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:19.884503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf00a42-4a70-46b0-a927-392d61f009f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:19.884837 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf00a42-4a70-46b0-a927-392d61f009f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:19.884945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523d9ca-eaa4-49da-b77b-d54f7fb90b78 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:19.886335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf00a42-4a70-46b0-a927-392d61f009f5 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:19.898334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 38/75] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:19.898832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf00a42-4a70-46b0-a927-392d61f009f5 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:19.919478 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd01e1-e5e1-4898-b0d6-a186a75fb206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:19.920098 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd01e1-e5e1-4898-b0d6-a186a75fb206 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:19.920380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd01e1-e5e1-4898-b0d6-a186a75fb206 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:19.920681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd01e1-e5e1-4898-b0d6-a186a75fb206 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:19.930609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd01e1-e5e1-4898-b0d6-a186a75fb206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-d5fd01e1-e5e1-4898-b0d6-a186a75fb206 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:19.936657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:19.937499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:19.938226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:19.938226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:19.938226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:19.938226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:19.938226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:19.938226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:19.938226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:19.938226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:19.938226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:19.938226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:19.938226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:19.938226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:19.938226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 38/76] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:19.939187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44662684-9f6d-41e6-8f57-3efaeb308737 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:19.939187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44662684-9f6d-41e6-8f57-3efaeb308737 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:19.939713 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44662684-9f6d-41e6-8f57-3efaeb308737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:19.956784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-44662684-9f6d-41e6-8f57-3efaeb308737 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:19.957967 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-44662684-9f6d-41e6-8f57-3efaeb308737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:19.998916 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf00a42-4a70-46b0-a927-392d61f009f5 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:20.024482 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 39/77] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:20.031597 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac9be7-b6bb-4eda-88f4-60ad4d0a5561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:20.032200 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac9be7-b6bb-4eda-88f4-60ad4d0a5561 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:20.032482 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac9be7-b6bb-4eda-88f4-60ad4d0a5561 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:20.032788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac9be7-b6bb-4eda-88f4-60ad4d0a5561 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:20.033451 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac9be7-b6bb-4eda-88f4-60ad4d0a5561 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:20.033539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44662684-9f6d-41e6-8f57-3efaeb308737 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:20.033761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac9be7-b6bb-4eda-88f4-60ad4d0a5561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:20.035138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac9be7-b6bb-4eda-88f4-60ad4d0a5561 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:20.036717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:20.044183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 39/78] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:00:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:20.044875 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac9be7-b6bb-4eda-88f4-60ad4d0a5561 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:20.093832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:20.138959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44662684-9f6d-41e6-8f57-3efaeb308737 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:20.147246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac9be7-b6bb-4eda-88f4-60ad4d0a5561 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:20.174693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6671b6-fc4e-48fa-bb62-70e22bc03416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:20.175437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6671b6-fc4e-48fa-bb62-70e22bc03416 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:20.175723 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6671b6-fc4e-48fa-bb62-70e22bc03416 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:20.176106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6671b6-fc4e-48fa-bb62-70e22bc03416 None admin] PATH_INFO: `/v3/users/1b637f155b514b58ab1c4d9c02381dc6` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:20.177270 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6671b6-fc4e-48fa-bb62-70e22bc03416 None admin] RBAC: Authorizing `identity:get_user(user_id=1b637f155b514b58ab1c4d9c02381dc6)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:20.178834 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6671b6-fc4e-48fa-bb62-70e22bc03416 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:20.185751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78a80b-fb5c-4c13-895f-44e71a46ce33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:20.185912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 40/79] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:20 2022] GET /identity/v3/users/1b637f155b514b58ab1c4d9c02381dc6 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:20.186000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78a80b-fb5c-4c13-895f-44e71a46ce33 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:20.186000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78a80b-fb5c-4c13-895f-44e71a46ce33 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:20.186130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6671b6-fc4e-48fa-bb62-70e22bc03416 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:20.186203 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78a80b-fb5c-4c13-895f-44e71a46ce33 None admin] PATH_INFO: `/v3/users/41b76a7e18b64d968720b907624136d4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:20.202748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78a80b-fb5c-4c13-895f-44e71a46ce33 None admin] RBAC: Authorizing `identity:get_user(user_id=41b76a7e18b64d968720b907624136d4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:20.204193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78a80b-fb5c-4c13-895f-44e71a46ce33 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:20.210361 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 40/80] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:20 2022] GET /identity/v3/users/41b76a7e18b64d968720b907624136d4 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:20.210361 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78a80b-fb5c-4c13-895f-44e71a46ce33 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:20.269612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:20.301464 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6671b6-fc4e-48fa-bb62-70e22bc03416 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:20.313883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78a80b-fb5c-4c13-895f-44e71a46ce33 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:20.322986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:20.335393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6d367-03ae-42a3-a618-8b6a26750aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:20.335998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6d367-03ae-42a3-a618-8b6a26750aae None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:20.336281 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6d367-03ae-42a3-a618-8b6a26750aae None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:20.336557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6d367-03ae-42a3-a618-8b6a26750aae None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:20.337784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6d367-03ae-42a3-a618-8b6a26750aae None admin] RBAC: Adding query filter params (name=member) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:20.338193 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6d367-03ae-42a3-a618-8b6a26750aae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:20.339445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6d367-03ae-42a3-a618-8b6a26750aae None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:20.347346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c15a8-edfe-40fa-9a3c-3fa4b199c92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:20.347506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 41/81] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:20.348147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c15a8-edfe-40fa-9a3c-3fa4b199c92d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:20.348246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6d367-03ae-42a3-a618-8b6a26750aae None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:20.348453 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c15a8-edfe-40fa-9a3c-3fa4b199c92d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:20.348734 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c15a8-edfe-40fa-9a3c-3fa4b199c92d None admin] PATH_INFO: `/v3/users/41b76a7e18b64d968720b907624136d4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:20.349874 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c15a8-edfe-40fa-9a3c-3fa4b199c92d None admin] RBAC: Authorizing `identity:get_user(user_id=41b76a7e18b64d968720b907624136d4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:20.351332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c15a8-edfe-40fa-9a3c-3fa4b199c92d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:20.358771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 41/82] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:20 2022] GET /identity/v3/users/41b76a7e18b64d968720b907624136d4 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:20.359602 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c15a8-edfe-40fa-9a3c-3fa4b199c92d None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:20.452922 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6d367-03ae-42a3-a618-8b6a26750aae None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:20.455206 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c15a8-edfe-40fa-9a3c-3fa4b199c92d None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:20.486087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abe173-5fe6-437a-b53d-998182a8fd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:20.486689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abe173-5fe6-437a-b53d-998182a8fd12 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:20.486974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abe173-5fe6-437a-b53d-998182a8fd12 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:20.487253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abe173-5fe6-437a-b53d-998182a8fd12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:20.488826 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355c2d0-53c2-4804-92e5-c651fad5eb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:20.489428 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355c2d0-53c2-4804-92e5-c651fad5eb82 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:20.489723 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355c2d0-53c2-4804-92e5-c651fad5eb82 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:20.489849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:20.489995 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355c2d0-53c2-4804-92e5-c651fad5eb82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:20.535395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:20.552936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7abe173-5fe6-437a-b53d-998182a8fd12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:20.554416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7abe173-5fe6-437a-b53d-998182a8fd12 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:20.558228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355c2d0-53c2-4804-92e5-c651fad5eb82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:20.559754 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355c2d0-53c2-4804-92e5-c651fad5eb82 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:20.649304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 42/83] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:20.650196 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abe173-5fe6-437a-b53d-998182a8fd12 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:20.657688 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 42/84] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:20.658368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355c2d0-53c2-4804-92e5-c651fad5eb82 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:20.684463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:20.733956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:20.749459 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abe173-5fe6-437a-b53d-998182a8fd12 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:20.755597 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355c2d0-53c2-4804-92e5-c651fad5eb82 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:20.784159 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46938300-f115-498a-a326-f605de87221b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:20.784818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46938300-f115-498a-a326-f605de87221b None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:20.785099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46938300-f115-498a-a326-f605de87221b None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:20.785374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46938300-f115-498a-a326-f605de87221b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:20.786500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46938300-f115-498a-a326-f605de87221b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:20.787998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46938300-f115-498a-a326-f605de87221b None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:20.789107 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df22c0-9f06-47be-82cc-5b401bcc63f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:20.789525 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 43/85] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:20.790373 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46938300-f115-498a-a326-f605de87221b None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:20.792912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df22c0-9f06-47be-82cc-5b401bcc63f2 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:20.793281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df22c0-9f06-47be-82cc-5b401bcc63f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:20.793646 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df22c0-9f06-47be-82cc-5b401bcc63f2 None admin] PATH_INFO: `/v3/users/41b76a7e18b64d968720b907624136d4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:20.794897 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df22c0-9f06-47be-82cc-5b401bcc63f2 None admin] RBAC: Authorizing `identity:get_user(user_id=41b76a7e18b64d968720b907624136d4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:20.796445 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df22c0-9f06-47be-82cc-5b401bcc63f2 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:20.798286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 43/86] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:20 2022] GET /identity/v3/users/41b76a7e18b64d968720b907624136d4 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:20.798919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df22c0-9f06-47be-82cc-5b401bcc63f2 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:20.895122 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46938300-f115-498a-a326-f605de87221b None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:20.896592 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:20.897780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df22c0-9f06-47be-82cc-5b401bcc63f2 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:20.928868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ee72d-02e7-44ea-ba28-ebcfb5f23147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:20.929511 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ee72d-02e7-44ea-ba28-ebcfb5f23147 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:20.929787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ee72d-02e7-44ea-ba28-ebcfb5f23147 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:20.930053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ee72d-02e7-44ea-ba28-ebcfb5f23147 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:20.931667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3bd8f-8b08-4c9a-afd4-bdc7ea5250b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:20.932537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3bd8f-8b08-4c9a-afd4-bdc7ea5250b8 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:20.932870 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3bd8f-8b08-4c9a-afd4-bdc7ea5250b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:20.933150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3bd8f-8b08-4c9a-afd4-bdc7ea5250b8 None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:20.934288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3bd8f-8b08-4c9a-afd4-bdc7ea5250b8 None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:20.935727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3bd8f-8b08-4c9a-afd4-bdc7ea5250b8 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:20.937381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 44/87] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:20 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:20.938144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3bd8f-8b08-4c9a-afd4-bdc7ea5250b8 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:20.952474 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:20.998526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ee72d-02e7-44ea-ba28-ebcfb5f23147 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:20.999930 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ee72d-02e7-44ea-ba28-ebcfb5f23147 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:21.035050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3bd8f-8b08-4c9a-afd4-bdc7ea5250b8 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:21.067874 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de241a-1a48-4fea-addb-7909435d7e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:21.068481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de241a-1a48-4fea-addb-7909435d7e61 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:21.068792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de241a-1a48-4fea-addb-7909435d7e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:21.069067 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de241a-1a48-4fea-addb-7909435d7e61 None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:21.070158 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de241a-1a48-4fea-addb-7909435d7e61 None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:21.072304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de241a-1a48-4fea-addb-7909435d7e61 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:21.073962 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 45/88] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:20 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:21.074693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de241a-1a48-4fea-addb-7909435d7e61 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:21.097795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 44/89] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:21.098639 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ee72d-02e7-44ea-ba28-ebcfb5f23147 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:21.100904 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:21.157222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:21.175873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de241a-1a48-4fea-addb-7909435d7e61 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:21.200304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ee72d-02e7-44ea-ba28-ebcfb5f23147 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:21.210923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a5170-c563-4030-a1e2-a0702161833e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:21.211528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a5170-c563-4030-a1e2-a0702161833e None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:21.211814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a5170-c563-4030-a1e2-a0702161833e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:21.212137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a5170-c563-4030-a1e2-a0702161833e None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:21.212902 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a5170-c563-4030-a1e2-a0702161833e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:21.214357 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a5170-c563-4030-a1e2-a0702161833e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:21.232915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789aded4-fd0c-48df-a2e1-a038d35d92c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:21.233520 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789aded4-fd0c-48df-a2e1-a038d35d92c1 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:21.233803 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789aded4-fd0c-48df-a2e1-a038d35d92c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:21.241371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789aded4-fd0c-48df-a2e1-a038d35d92c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:21.295772 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 46/90] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:00:21 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:00:21.296618 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a5170-c563-4030-a1e2-a0702161833e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:21.301493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789aded4-fd0c-48df-a2e1-a038d35d92c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:21.303252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789aded4-fd0c-48df-a2e1-a038d35d92c1 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:21.306725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:21.424130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 45/91] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:21.424324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a5170-c563-4030-a1e2-a0702161833e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:21.424430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789aded4-fd0c-48df-a2e1-a038d35d92c1 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:21.463055 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c435f0-100a-47d4-87a4-3ec250fc3e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:21.463736 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c435f0-100a-47d4-87a4-3ec250fc3e6e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:21.464025 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c435f0-100a-47d4-87a4-3ec250fc3e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:21.464298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c435f0-100a-47d4-87a4-3ec250fc3e6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:21.465105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c435f0-100a-47d4-87a4-3ec250fc3e6e None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=4fe97b975d834d83a6733a0b2591bba4, user.id=1b637f155b514b58ab1c4d9c02381dc6) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:21.465414 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c435f0-100a-47d4-87a4-3ec250fc3e6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:21.466867 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c435f0-100a-47d4-87a4-3ec250fc3e6e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:21.473055 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:21.476410 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 47/92] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:21 2022] GET /identity/v3/role_assignments?user.id=1b637f155b514b58ab1c4d9c02381dc6&scope.project.id=4fe97b975d834d83a6733a0b2591bba4&role.id=906e3a8a6c7540a38f629f2567133347 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:21.477356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c435f0-100a-47d4-87a4-3ec250fc3e6e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:21.563393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789aded4-fd0c-48df-a2e1-a038d35d92c1 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:21.566227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:21.617990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca92fb-174d-4188-8b9b-1b851c5839b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:21.619353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca92fb-174d-4188-8b9b-1b851c5839b7 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:21.619752 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca92fb-174d-4188-8b9b-1b851c5839b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:21.620524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca92fb-174d-4188-8b9b-1b851c5839b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:21.621430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca92fb-174d-4188-8b9b-1b851c5839b7 None admin] RBAC: Adding query filter params (role.id=8b2ee9b11a934b67aa3fd68c497fbdf7, scope.project.id=4fe97b975d834d83a6733a0b2591bba4, user.id=41b76a7e18b64d968720b907624136d4) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:21.621845 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca92fb-174d-4188-8b9b-1b851c5839b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:21.623609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca92fb-174d-4188-8b9b-1b851c5839b7 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:21.631819 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c435f0-100a-47d4-87a4-3ec250fc3e6e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:21.632380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 46/93] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:21 2022] GET /identity/v3/role_assignments?user.id=41b76a7e18b64d968720b907624136d4&scope.project.id=4fe97b975d834d83a6733a0b2591bba4&role.id=8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:21.633282 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca92fb-174d-4188-8b9b-1b851c5839b7 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:21.688971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f51baf-5549-4867-9695-73ebda1b799d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:21.689632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f51baf-5549-4867-9695-73ebda1b799d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:21.689921 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f51baf-5549-4867-9695-73ebda1b799d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:21.690203 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f51baf-5549-4867-9695-73ebda1b799d None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:21.692133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f51baf-5549-4867-9695-73ebda1b799d None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:21.693540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f51baf-5549-4867-9695-73ebda1b799d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:21.697254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 48/94] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:21 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:21.707969 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f51baf-5549-4867-9695-73ebda1b799d None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:21.765987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca92fb-174d-4188-8b9b-1b851c5839b7 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:21.804143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e44c4-5349-42a1-9a23-c913c5988c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:21.805057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e44c4-5349-42a1-9a23-c913c5988c94 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:21.805168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:21.805567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e44c4-5349-42a1-9a23-c913c5988c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:21.807160 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e44c4-5349-42a1-9a23-c913c5988c94 None admin] PATH_INFO: `/v3/projects/b27e60e99095427f972d99e4d3b16c9c` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:21.808714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e44c4-5349-42a1-9a23-c913c5988c94 None admin] RBAC: Authorizing `identity:get_project(project_id=b27e60e99095427f972d99e4d3b16c9c)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:21.810744 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e44c4-5349-42a1-9a23-c913c5988c94 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:21.812561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 47/95] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:21 2022] GET /identity/v3/projects/b27e60e99095427f972d99e4d3b16c9c => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:21.822382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e44c4-5349-42a1-9a23-c913c5988c94 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:21.845848 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f51baf-5549-4867-9695-73ebda1b799d None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:21.863951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:21.914530 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044362c8-9a39-4677-b027-e26261a0c769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:21.915829 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044362c8-9a39-4677-b027-e26261a0c769 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:21.916291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044362c8-9a39-4677-b027-e26261a0c769 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:21.916850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044362c8-9a39-4677-b027-e26261a0c769 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:21.917871 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044362c8-9a39-4677-b027-e26261a0c769 None admin] RBAC: Adding query filter params (role.id=b5e411fc96434df69c6cc1df951b3e6f, scope.project.id=4fe97b975d834d83a6733a0b2591bba4, user.id=41b76a7e18b64d968720b907624136d4) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:21.918358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044362c8-9a39-4677-b027-e26261a0c769 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:21.919969 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044362c8-9a39-4677-b027-e26261a0c769 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:21.926657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 49/96] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:21 2022] GET /identity/v3/role_assignments?user.id=41b76a7e18b64d968720b907624136d4&scope.project.id=4fe97b975d834d83a6733a0b2591bba4&role.id=b5e411fc96434df69c6cc1df951b3e6f => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:21.934177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e44c4-5349-42a1-9a23-c913c5988c94 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:21.974757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaccd86-1d3e-4631-addc-3935db729a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:21.975528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaccd86-1d3e-4631-addc-3935db729a46 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:21.975884 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaccd86-1d3e-4631-addc-3935db729a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:21.976225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaccd86-1d3e-4631-addc-3935db729a46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:21.977087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaccd86-1d3e-4631-addc-3935db729a46 None admin] RBAC: Adding query filter params (role.id=8b2ee9b11a934b67aa3fd68c497fbdf7, scope.project.id=b27e60e99095427f972d99e4d3b16c9c, user.id=41b76a7e18b64d968720b907624136d4) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:21.977514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaccd86-1d3e-4631-addc-3935db729a46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:21.979654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaccd86-1d3e-4631-addc-3935db729a46 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:21.986430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 48/97] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:21 2022] GET /identity/v3/role_assignments?user.id=41b76a7e18b64d968720b907624136d4&scope.project.id=b27e60e99095427f972d99e4d3b16c9c&role.id=8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:24.190033 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93638cb-4dfb-4924-bc09-85ecb07d59b0 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:24.190552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93638cb-4dfb-4924-bc09-85ecb07d59b0 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:24.190631 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93638cb-4dfb-4924-bc09-85ecb07d59b0 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:24.191957 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 50/98] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:24.197484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645920c4-f9a9-40d0-a747-91eddaccd4ca None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:24.197919 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645920c4-f9a9-40d0-a747-91eddaccd4ca None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:24.198047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645920c4-f9a9-40d0-a747-91eddaccd4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:24.216526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-645920c4-f9a9-40d0-a747-91eddaccd4ca None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:24.217768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-645920c4-f9a9-40d0-a747-91eddaccd4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:24.280603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa40887-adfb-4ffb-b869-f65ab3d43e25 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:24.280919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa40887-adfb-4ffb-b869-f65ab3d43e25 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:24.281195 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa40887-adfb-4ffb-b869-f65ab3d43e25 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:24.282693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 51/99] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:00:24.293764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 49/100] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:24.297472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bd4b3-83dd-49df-8bf2-55c54a3eec6d None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:24.297818 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bd4b3-83dd-49df-8bf2-55c54a3eec6d None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:24.298105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bd4b3-83dd-49df-8bf2-55c54a3eec6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:24.321476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-145bd4b3-83dd-49df-8bf2-55c54a3eec6d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:24.322683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-145bd4b3-83dd-49df-8bf2-55c54a3eec6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:24.393883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 52/101] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:24.558278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacc62b-33d5-463b-abcd-0893092343b2 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:24.558578 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacc62b-33d5-463b-abcd-0893092343b2 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:24.558759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacc62b-33d5-463b-abcd-0893092343b2 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:24.560244 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 50/102] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:24.570092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d744afc-0c7f-42f0-ace6-4a75e647f4e8 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:24.570393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d744afc-0c7f-42f0-ace6-4a75e647f4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:24.570676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d744afc-0c7f-42f0-ace6-4a75e647f4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:24.601614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-5d744afc-0c7f-42f0-ace6-4a75e647f4e8 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:24.603352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-5d744afc-0c7f-42f0-ace6-4a75e647f4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:24.622158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de411d6-3c26-4e4a-a31d-8bb966820f05 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:24.622460 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de411d6-3c26-4e4a-a31d-8bb966820f05 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:24.622730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de411d6-3c26-4e4a-a31d-8bb966820f05 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:24.624243 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 51/103] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:24.629332 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de02490-b1df-432e-bdef-83aee125d171 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:24.629622 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de02490-b1df-432e-bdef-83aee125d171 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:24.629905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de02490-b1df-432e-bdef-83aee125d171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:24.662174 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-7de02490-b1df-432e-bdef-83aee125d171 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:24.663382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7de02490-b1df-432e-bdef-83aee125d171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:24.713014 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:24.717866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:24.729162 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 53/104] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:24.730143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d744afc-0c7f-42f0-ace6-4a75e647f4e8 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:24.759724 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 52/105] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:00:24.761057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52141bb7-16e5-4617-99f9-5b0a4b8259ac None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:24.761352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52141bb7-16e5-4617-99f9-5b0a4b8259ac None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:24.761635 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52141bb7-16e5-4617-99f9-5b0a4b8259ac None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:24.762974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 53/106] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:24.773976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f6226-91a8-4d69-984a-878395e3bbb0 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:24.774798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f6226-91a8-4d69-984a-878395e3bbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:24.775151 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f6226-91a8-4d69-984a-878395e3bbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:24.813798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-798f6226-91a8-4d69-984a-878395e3bbb0 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:24.815036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-798f6226-91a8-4d69-984a-878395e3bbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:24.848220 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d744afc-0c7f-42f0-ace6-4a75e647f4e8 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:24.916328 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2ea3f-a3b0-4366-b63b-10748f6d1f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:24.916992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2ea3f-a3b0-4366-b63b-10748f6d1f8b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:24.917278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2ea3f-a3b0-4366-b63b-10748f6d1f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:24.917560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2ea3f-a3b0-4366-b63b-10748f6d1f8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:24.919279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 54/107] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:24.920137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f6226-91a8-4d69-984a-878395e3bbb0 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:24.921168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2ea3f-a3b0-4366-b63b-10748f6d1f8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:24.922651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2ea3f-a3b0-4366-b63b-10748f6d1f8b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:24.924237 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 54/108] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:24.934024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2ea3f-a3b0-4366-b63b-10748f6d1f8b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:25.046697 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:25.062856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f6226-91a8-4d69-984a-878395e3bbb0 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:25.089709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:25.104466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff208bf1-6ee5-4ed4-9da7-ffc26f1f3f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:25.105133 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff208bf1-6ee5-4ed4-9da7-ffc26f1f3f3c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:25.105420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff208bf1-6ee5-4ed4-9da7-ffc26f1f3f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:25.105696 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff208bf1-6ee5-4ed4-9da7-ffc26f1f3f3c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:25.128482 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff208bf1-6ee5-4ed4-9da7-ffc26f1f3f3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-ff208bf1-6ee5-4ed4-9da7-ffc26f1f3f3c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:25.139370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:25.140210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:25.141018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:25.141018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:25.141018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:25.141018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:25.141018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:25.141018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:25.141018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:25.141018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:25.141018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:25.141018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:25.141018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:25.141018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:25.141018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 55/109] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:25.141674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff208bf1-6ee5-4ed4-9da7-ffc26f1f3f3c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:25.154846 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2ea3f-a3b0-4366-b63b-10748f6d1f8b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:25.199397 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a9bd9-a342-4c50-b9bc-9c8229722b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:25.201306 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a9bd9-a342-4c50-b9bc-9c8229722b6e None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:25.201783 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a9bd9-a342-4c50-b9bc-9c8229722b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:25.202675 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a9bd9-a342-4c50-b9bc-9c8229722b6e None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:25.203827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a9bd9-a342-4c50-b9bc-9c8229722b6e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:25.206198 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a9bd9-a342-4c50-b9bc-9c8229722b6e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:25.264619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff208bf1-6ee5-4ed4-9da7-ffc26f1f3f3c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:25.297832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 55/110] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 12 12:00:24 2022] POST /identity/v3/projects => generated 302 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:00:25.298110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b951205-7796-427c-a766-05d35f4621a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:25.298742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b951205-7796-427c-a766-05d35f4621a2 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:25.298925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a9bd9-a342-4c50-b9bc-9c8229722b6e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:25.299107 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b951205-7796-427c-a766-05d35f4621a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:25.299385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b951205-7796-427c-a766-05d35f4621a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:25.312044 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b951205-7796-427c-a766-05d35f4621a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-4b951205-7796-427c-a766-05d35f4621a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:25.317568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:25.318463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:25.319208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:25.319208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:25.319208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:25.319208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:25.319208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:25.319208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:25.319208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:25.319208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:25.319208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:25.319208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:25.319208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:25.319208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:25.319208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 56/111] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:00:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:25.319904 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b951205-7796-427c-a766-05d35f4621a2 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:25.334866 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:25.418496 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a9bd9-a342-4c50-b9bc-9c8229722b6e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:25.434134 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:25.436835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b951205-7796-427c-a766-05d35f4621a2 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:25.457661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c368b-c093-43c2-8a5c-69e428ec0be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:25.458370 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c368b-c093-43c2-8a5c-69e428ec0be1 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:25.458665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c368b-c093-43c2-8a5c-69e428ec0be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:25.458942 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c368b-c093-43c2-8a5c-69e428ec0be1 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:25.460019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c368b-c093-43c2-8a5c-69e428ec0be1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:25.460019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c368b-c093-43c2-8a5c-69e428ec0be1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:25.461744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c368b-c093-43c2-8a5c-69e428ec0be1 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:25.471363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 56/112] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:25.472775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe2334-f54b-485f-834c-5a9f5d980821 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:25.473035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe2334-f54b-485f-834c-5a9f5d980821 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:25.473301 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe2334-f54b-485f-834c-5a9f5d980821 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:25.474713 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 57/113] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:25.475243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe2334-f54b-485f-834c-5a9f5d980821 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:25.476156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daec5c-6caa-46aa-b685-924247821152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:25.476844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daec5c-6caa-46aa-b685-924247821152 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:25.477130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daec5c-6caa-46aa-b685-924247821152 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:25.477639 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daec5c-6caa-46aa-b685-924247821152 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:25.489286 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4daec5c-6caa-46aa-b685-924247821152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-e4daec5c-6caa-46aa-b685-924247821152 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:25.495441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:25.496467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:25.497522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:25.497522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:25.497522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:25.497522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:25.497522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:25.497522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:25.497522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:25.497522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:25.497522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:25.497522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:25.497522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:25.497522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:25.497522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 57/114] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:00:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:25.498262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daec5c-6caa-46aa-b685-924247821152 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:25.562483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:25.596009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daec5c-6caa-46aa-b685-924247821152 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:25.596945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe2334-f54b-485f-834c-5a9f5d980821 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:25.634639 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b77e1a-5a73-4d4b-833f-89aefbe6956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:25.634979 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272254f9-1d9d-45bc-8aee-eb6c09e57e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:25.635253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b77e1a-5a73-4d4b-833f-89aefbe6956c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:25.635531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b77e1a-5a73-4d4b-833f-89aefbe6956c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:25.635645 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272254f9-1d9d-45bc-8aee-eb6c09e57e3f None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:25.635795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b77e1a-5a73-4d4b-833f-89aefbe6956c None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:25.635942 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272254f9-1d9d-45bc-8aee-eb6c09e57e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:25.636216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272254f9-1d9d-45bc-8aee-eb6c09e57e3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:25.636467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b77e1a-5a73-4d4b-833f-89aefbe6956c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:25.636803 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b77e1a-5a73-4d4b-833f-89aefbe6956c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:25.638187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b77e1a-5a73-4d4b-833f-89aefbe6956c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:25.646716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 58/115] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:25.647352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b77e1a-5a73-4d4b-833f-89aefbe6956c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:25.653325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272254f9-1d9d-45bc-8aee-eb6c09e57e3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-272254f9-1d9d-45bc-8aee-eb6c09e57e3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:25.658091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:25.658956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:25.659716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:25.659716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:25.659716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:25.659716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:25.659716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:25.659716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:25.659716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:25.659716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:25.659716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:25.659716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:25.659716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:25.659716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:25.659716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 58/116] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:25.661081 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3cc93-ff68-46d3-89a1-a13236a25b13 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:25.661369 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3cc93-ff68-46d3-89a1-a13236a25b13 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:25.661646 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3cc93-ff68-46d3-89a1-a13236a25b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:25.679024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-36b3cc93-ff68-46d3-89a1-a13236a25b13 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:25.680224 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-36b3cc93-ff68-46d3-89a1-a13236a25b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:25.709723 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:25.752796 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 59/117] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:25.760799 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3cc93-ff68-46d3-89a1-a13236a25b13 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:25.769367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b77e1a-5a73-4d4b-833f-89aefbe6956c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:25.811137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:25.816112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a8aa7-779b-4d99-b054-73db5a3a849e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:25.816915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a8aa7-779b-4d99-b054-73db5a3a849e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:25.817271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a8aa7-779b-4d99-b054-73db5a3a849e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:25.817661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a8aa7-779b-4d99-b054-73db5a3a849e None admin] PATH_INFO: `/v3/users/aaf97666c16d4c7684bda4f48b7cd610` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:25.834363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a8aa7-779b-4d99-b054-73db5a3a849e None admin] RBAC: Authorizing `identity:get_user(user_id=aaf97666c16d4c7684bda4f48b7cd610)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:25.835891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a8aa7-779b-4d99-b054-73db5a3a849e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:25.837687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 59/118] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:25 2022] GET /identity/v3/users/aaf97666c16d4c7684bda4f48b7cd610 => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:25.838548 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a8aa7-779b-4d99-b054-73db5a3a849e None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:25.897427 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3cc93-ff68-46d3-89a1-a13236a25b13 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:25.936812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68016da9-5e98-4344-8fa0-ec29b051e3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:25.937522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68016da9-5e98-4344-8fa0-ec29b051e3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:25.937814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68016da9-5e98-4344-8fa0-ec29b051e3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:25.938112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68016da9-5e98-4344-8fa0-ec29b051e3f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:25.938776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68016da9-5e98-4344-8fa0-ec29b051e3f0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:25.939083 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68016da9-5e98-4344-8fa0-ec29b051e3f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:25.940609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68016da9-5e98-4344-8fa0-ec29b051e3f0 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:25.951113 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 60/119] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:00:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:25.952074 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68016da9-5e98-4344-8fa0-ec29b051e3f0 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:25.983481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:25.987906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a8aa7-779b-4d99-b054-73db5a3a849e None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:26.036391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f18dd3-c887-4051-a2f7-786e4b72382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:26.037120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f18dd3-c887-4051-a2f7-786e4b72382e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:26.037409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f18dd3-c887-4051-a2f7-786e4b72382e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:26.037687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f18dd3-c887-4051-a2f7-786e4b72382e None admin] PATH_INFO: `/v3/users/1b637f155b514b58ab1c4d9c02381dc6` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:26.038885 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f18dd3-c887-4051-a2f7-786e4b72382e None admin] RBAC: Authorizing `identity:get_user(user_id=1b637f155b514b58ab1c4d9c02381dc6)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:26.043381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f18dd3-c887-4051-a2f7-786e4b72382e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:26.045074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 60/120] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:25 2022] GET /identity/v3/users/1b637f155b514b58ab1c4d9c02381dc6 => generated 247 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:26.045871 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f18dd3-c887-4051-a2f7-786e4b72382e None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:26.071920 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68016da9-5e98-4344-8fa0-ec29b051e3f0 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:26.107722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc95bf-ed24-4d8f-85dd-5daba4ebdc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:26.108314 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc95bf-ed24-4d8f-85dd-5daba4ebdc0b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:26.108595 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc95bf-ed24-4d8f-85dd-5daba4ebdc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:26.108896 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc95bf-ed24-4d8f-85dd-5daba4ebdc0b None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:26.109552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc95bf-ed24-4d8f-85dd-5daba4ebdc0b None admin] RBAC: Adding query filter params (name=member) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:26.109854 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc95bf-ed24-4d8f-85dd-5daba4ebdc0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:26.111217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc95bf-ed24-4d8f-85dd-5daba4ebdc0b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:26.114810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:26.120225 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 61/121] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:26.124844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc95bf-ed24-4d8f-85dd-5daba4ebdc0b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:26.164156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f18dd3-c887-4051-a2f7-786e4b72382e None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:26.202091 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3aebd5-f645-41fa-a009-e65a584cbe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:26.202822 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3aebd5-f645-41fa-a009-e65a584cbe2b None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:26.203334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3aebd5-f645-41fa-a009-e65a584cbe2b None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:26.203461 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3aebd5-f645-41fa-a009-e65a584cbe2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:26.216999 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:26.229959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc95bf-ed24-4d8f-85dd-5daba4ebdc0b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:26.264467 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39916363-7405-4331-a915-215cf716168c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:26.265122 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39916363-7405-4331-a915-215cf716168c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:26.265408 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39916363-7405-4331-a915-215cf716168c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:26.265684 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39916363-7405-4331-a915-215cf716168c None admin] PATH_INFO: `/v3/users/41b76a7e18b64d968720b907624136d4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:26.266810 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39916363-7405-4331-a915-215cf716168c None admin] RBAC: Authorizing `identity:get_user(user_id=41b76a7e18b64d968720b907624136d4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:26.268245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39916363-7405-4331-a915-215cf716168c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:26.270033 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 62/122] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:26 2022] GET /identity/v3/users/41b76a7e18b64d968720b907624136d4 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:26.275511 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39916363-7405-4331-a915-215cf716168c None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:26.277613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3aebd5-f645-41fa-a009-e65a584cbe2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:26.279084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3aebd5-f645-41fa-a009-e65a584cbe2b None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:26.342627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:26.379446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39916363-7405-4331-a915-215cf716168c None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:26.390562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 61/123] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:26.391861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3aebd5-f645-41fa-a009-e65a584cbe2b None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:26.414497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23c7de-d6f8-4be4-b851-4cb292edf23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:26.415123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23c7de-d6f8-4be4-b851-4cb292edf23d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:26.415410 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23c7de-d6f8-4be4-b851-4cb292edf23d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:26.415683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23c7de-d6f8-4be4-b851-4cb292edf23d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:26.426433 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23c7de-d6f8-4be4-b851-4cb292edf23d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-0c23c7de-d6f8-4be4-b851-4cb292edf23d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:26.432599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:26.433537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:26.434352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:26.434352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:26.434352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:26.434352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:26.434352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:26.434352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:26.434352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:26.434352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:26.434352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:26.434352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:26.434352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:26.434352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:26.434352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 63/124] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:26.434970 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23c7de-d6f8-4be4-b851-4cb292edf23d None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:26.455555 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:26.509386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3aebd5-f645-41fa-a009-e65a584cbe2b None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:26.539937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23c7de-d6f8-4be4-b851-4cb292edf23d None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:26.550718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417c2f6-d770-41bb-88f6-515bc9357ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:26.551511 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417c2f6-d770-41bb-88f6-515bc9357ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:26.551935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417c2f6-d770-41bb-88f6-515bc9357ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:26.552338 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417c2f6-d770-41bb-88f6-515bc9357ab7 None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:26.553622 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417c2f6-d770-41bb-88f6-515bc9357ab7 None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:26.555303 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417c2f6-d770-41bb-88f6-515bc9357ab7 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:26.557855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 62/125] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:26 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:26.558763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417c2f6-d770-41bb-88f6-515bc9357ab7 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:26.582085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6d775-02cc-4be3-a094-d60555db66f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:26.582711 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6d775-02cc-4be3-a094-d60555db66f5 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:26.582998 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6d775-02cc-4be3-a094-d60555db66f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:26.583279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6d775-02cc-4be3-a094-d60555db66f5 None admin] PATH_INFO: `/v3/users/41b76a7e18b64d968720b907624136d4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:26.584408 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6d775-02cc-4be3-a094-d60555db66f5 None admin] RBAC: Authorizing `identity:get_user(user_id=41b76a7e18b64d968720b907624136d4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:26.585898 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6d775-02cc-4be3-a094-d60555db66f5 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:26.587676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 64/126] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:26 2022] GET /identity/v3/users/41b76a7e18b64d968720b907624136d4 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:26.588359 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6d775-02cc-4be3-a094-d60555db66f5 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:26.593330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:26.670540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417c2f6-d770-41bb-88f6-515bc9357ab7 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:26.676222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:26.693496 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6d775-02cc-4be3-a094-d60555db66f5 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:26.705419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9de32-191f-48be-8b3c-9eb56ff1c09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:26.706057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9de32-191f-48be-8b3c-9eb56ff1c09d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:26.706383 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9de32-191f-48be-8b3c-9eb56ff1c09d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:26.706660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9de32-191f-48be-8b3c-9eb56ff1c09d None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:26.707969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9de32-191f-48be-8b3c-9eb56ff1c09d None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:26.709482 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9de32-191f-48be-8b3c-9eb56ff1c09d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:26.711302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 63/127] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:26 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:26.716849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9de32-191f-48be-8b3c-9eb56ff1c09d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:26.730471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d266b-7138-4178-a07f-7ee4eeb7e5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:26.731329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d266b-7138-4178-a07f-7ee4eeb7e5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:26.731894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d266b-7138-4178-a07f-7ee4eeb7e5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:26.732460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d266b-7138-4178-a07f-7ee4eeb7e5c3 None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:26.733632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d266b-7138-4178-a07f-7ee4eeb7e5c3 None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:26.735204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d266b-7138-4178-a07f-7ee4eeb7e5c3 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:26.737270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 65/128] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:26 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:26.737840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d266b-7138-4178-a07f-7ee4eeb7e5c3 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:26.822878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:26.831491 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9de32-191f-48be-8b3c-9eb56ff1c09d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:26.846403 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d266b-7138-4178-a07f-7ee4eeb7e5c3 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:26.870396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0fcb8-a29d-4d8e-9ddd-be3613cdef13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:26.871010 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0fcb8-a29d-4d8e-9ddd-be3613cdef13 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:26.871297 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0fcb8-a29d-4d8e-9ddd-be3613cdef13 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:26.871572 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0fcb8-a29d-4d8e-9ddd-be3613cdef13 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:26.872250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0fcb8-a29d-4d8e-9ddd-be3613cdef13 None admin] RBAC: Adding query filter params (name=member) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:26.872551 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0fcb8-a29d-4d8e-9ddd-be3613cdef13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:26.873975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0fcb8-a29d-4d8e-9ddd-be3613cdef13 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:26.882629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 64/129] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:26.883353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0fcb8-a29d-4d8e-9ddd-be3613cdef13 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:26.888192 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a889c-bf87-4f01-8813-53fa2c40221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:26.888896 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a889c-bf87-4f01-8813-53fa2c40221c None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:26.889187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a889c-bf87-4f01-8813-53fa2c40221c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:26.889467 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a889c-bf87-4f01-8813-53fa2c40221c None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4/users/1b637f155b514b58ab1c4d9c02381dc6/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:26.891402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a889c-bf87-4f01-8813-53fa2c40221c None admin] RBAC: Authorizing `identity:create_grant(project_id=4fe97b975d834d83a6733a0b2591bba4, user_id=1b637f155b514b58ab1c4d9c02381dc6, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:26.892885 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a889c-bf87-4f01-8813-53fa2c40221c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:26.970552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 66/130] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:00:26 2022] PUT /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4/users/1b637f155b514b58ab1c4d9c02381dc6/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:00:26.971774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a889c-bf87-4f01-8813-53fa2c40221c None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:26.989208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:27.005163 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0fcb8-a29d-4d8e-9ddd-be3613cdef13 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:27.059045 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232554ff-38cf-4d77-a8cb-0098571b97f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:27.060355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232554ff-38cf-4d77-a8cb-0098571b97f3 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:27.061795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232554ff-38cf-4d77-a8cb-0098571b97f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:27.062344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232554ff-38cf-4d77-a8cb-0098571b97f3 None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:27.063831 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232554ff-38cf-4d77-a8cb-0098571b97f3 None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:27.065840 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232554ff-38cf-4d77-a8cb-0098571b97f3 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:27.069846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 65/131] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:26 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:27.072014 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232554ff-38cf-4d77-a8cb-0098571b97f3 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:27.098834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a889c-bf87-4f01-8813-53fa2c40221c None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:27.111142 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:27.146113 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe4ff7-69aa-40df-9706-e9345242dab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:27.146783 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe4ff7-69aa-40df-9706-e9345242dab8 None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:27.147075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe4ff7-69aa-40df-9706-e9345242dab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:27.147359 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe4ff7-69aa-40df-9706-e9345242dab8 None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4/users/41b76a7e18b64d968720b907624136d4/roles/b5e411fc96434df69c6cc1df951b3e6f` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:27.150956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe4ff7-69aa-40df-9706-e9345242dab8 None admin] RBAC: Authorizing `identity:create_grant(project_id=4fe97b975d834d83a6733a0b2591bba4, user_id=41b76a7e18b64d968720b907624136d4, role_id=b5e411fc96434df69c6cc1df951b3e6f)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:27.152743 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe4ff7-69aa-40df-9706-e9345242dab8 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:27.204265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232554ff-38cf-4d77-a8cb-0098571b97f3 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:27.244121 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 67/132] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:00:26 2022] PUT /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4/users/41b76a7e18b64d968720b907624136d4/roles/b5e411fc96434df69c6cc1df951b3e6f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:00:27.252889 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe4ff7-69aa-40df-9706-e9345242dab8 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:27.255856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998634a0-9a06-46c1-877b-53a0df86bfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:27.256781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998634a0-9a06-46c1-877b-53a0df86bfb5 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:27.257225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998634a0-9a06-46c1-877b-53a0df86bfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:27.257678 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998634a0-9a06-46c1-877b-53a0df86bfb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:27.259198 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998634a0-9a06-46c1-877b-53a0df86bfb5 None admin] RBAC: Adding query filter params (role.id=bf2de8c80c5e4be79e1ea7adc3cbfc56, scope.project.id=4fe97b975d834d83a6733a0b2591bba4, user.id=aaf97666c16d4c7684bda4f48b7cd610) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:27.259751 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998634a0-9a06-46c1-877b-53a0df86bfb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:27.261818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998634a0-9a06-46c1-877b-53a0df86bfb5 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:27.268675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 66/133] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:27 2022] GET /identity/v3/role_assignments?user.id=aaf97666c16d4c7684bda4f48b7cd610&scope.project.id=4fe97b975d834d83a6733a0b2591bba4&role.id=bf2de8c80c5e4be79e1ea7adc3cbfc56 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:27.269875 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998634a0-9a06-46c1-877b-53a0df86bfb5 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:27.424351 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:27.443487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998634a0-9a06-46c1-877b-53a0df86bfb5 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:27.449714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:27.486277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dc9dd-3987-4ca4-b14d-5f0734b04827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:27.486947 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dc9dd-3987-4ca4-b14d-5f0734b04827 None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:27.487232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dc9dd-3987-4ca4-b14d-5f0734b04827 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:27.487511 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dc9dd-3987-4ca4-b14d-5f0734b04827 None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4/users/41b76a7e18b64d968720b907624136d4/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:27.489829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997dc9dd-3987-4ca4-b14d-5f0734b04827 None admin] RBAC: Authorizing `identity:create_grant(project_id=4fe97b975d834d83a6733a0b2591bba4, user_id=41b76a7e18b64d968720b907624136d4, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:27.491291 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997dc9dd-3987-4ca4-b14d-5f0734b04827 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:27.493150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe4ff7-69aa-40df-9706-e9345242dab8 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:27.531450 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce5ab7-65e8-449d-b049-e046753f6ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:27.532433 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce5ab7-65e8-449d-b049-e046753f6ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:27.532956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce5ab7-65e8-449d-b049-e046753f6ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:27.533441 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce5ab7-65e8-449d-b049-e046753f6ec9 None admin] PATH_INFO: `/v3/users/41b76a7e18b64d968720b907624136d4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:27.534791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce5ab7-65e8-449d-b049-e046753f6ec9 None admin] RBAC: Authorizing `identity:get_user(user_id=41b76a7e18b64d968720b907624136d4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:27.536439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce5ab7-65e8-449d-b049-e046753f6ec9 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:27.538288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 68/134] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:27 2022] GET /identity/v3/users/41b76a7e18b64d968720b907624136d4 => generated 275 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:27.540432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d4bed-a207-4089-96e4-607b88413daa None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:27.540952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d4bed-a207-4089-96e4-607b88413daa None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:27.541498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d4bed-a207-4089-96e4-607b88413daa None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:27.542966 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 69/135] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:27.548307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9cb61-95b7-4fc3-8cea-36ccb4f384e3 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:27.548929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9cb61-95b7-4fc3-8cea-36ccb4f384e3 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:27.549417 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9cb61-95b7-4fc3-8cea-36ccb4f384e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:27.570339 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-3df9cb61-95b7-4fc3-8cea-36ccb4f384e3 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:27.572717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-3df9cb61-95b7-4fc3-8cea-36ccb4f384e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:27.580934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 67/136] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:00:27 2022] PUT /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4/users/41b76a7e18b64d968720b907624136d4/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:00:27.581535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dc9dd-3987-4ca4-b14d-5f0734b04827 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:27.701176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 70/137] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:00:27.820855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dc9dd-3987-4ca4-b14d-5f0734b04827 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:27.875753 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f4ef6-29b4-4e84-baaf-b10e916084fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:27.876393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f4ef6-29b4-4e84-baaf-b10e916084fd None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:27.876696 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f4ef6-29b4-4e84-baaf-b10e916084fd None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:27.876979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f4ef6-29b4-4e84-baaf-b10e916084fd None admin] PATH_INFO: `/v3/projects/b27e60e99095427f972d99e4d3b16c9c` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:27.878113 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f4ef6-29b4-4e84-baaf-b10e916084fd None admin] RBAC: Authorizing `identity:get_project(project_id=b27e60e99095427f972d99e4d3b16c9c)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:27.879573 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f4ef6-29b4-4e84-baaf-b10e916084fd None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:27.881225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 68/138] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:27 2022] GET /identity/v3/projects/b27e60e99095427f972d99e4d3b16c9c => generated 312 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:27.887172 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9cb61-95b7-4fc3-8cea-36ccb4f384e3 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:27.993238 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:28.004935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f4ef6-29b4-4e84-baaf-b10e916084fd None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:28.033521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:28.042055 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9cb61-95b7-4fc3-8cea-36ccb4f384e3 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:28.110535 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc38c64-d7c4-4c5e-a207-777c95b0cba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:28.111135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc38c64-d7c4-4c5e-a207-777c95b0cba7 None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:28.111417 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc38c64-d7c4-4c5e-a207-777c95b0cba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:28.111692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc38c64-d7c4-4c5e-a207-777c95b0cba7 None admin] PATH_INFO: `/v3/projects/b27e60e99095427f972d99e4d3b16c9c/users/41b76a7e18b64d968720b907624136d4/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:28.114926 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc38c64-d7c4-4c5e-a207-777c95b0cba7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b27e60e99095427f972d99e4d3b16c9c, user_id=41b76a7e18b64d968720b907624136d4, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:28.116622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc38c64-d7c4-4c5e-a207-777c95b0cba7 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:28.164323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f4ef6-29b4-4e84-baaf-b10e916084fd None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:28.193185 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 71/139] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:00:27 2022] PUT /identity/v3/projects/b27e60e99095427f972d99e4d3b16c9c/users/41b76a7e18b64d968720b907624136d4/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:00:28.208062 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7f81b-d547-4642-9445-e17f97ea0ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:28.208739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7f81b-d547-4642-9445-e17f97ea0ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:28.209065 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7f81b-d547-4642-9445-e17f97ea0ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:28.209353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7f81b-d547-4642-9445-e17f97ea0ad9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:28.210529 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7f81b-d547-4642-9445-e17f97ea0ad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:28.212261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7f81b-d547-4642-9445-e17f97ea0ad9 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:28.214011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 69/140] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:28.220240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc38c64-d7c4-4c5e-a207-777c95b0cba7 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:28.349545 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc38c64-d7c4-4c5e-a207-777c95b0cba7 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:28.390583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:28.397032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e3ab6-c1fc-4ee5-a298-7fc47c660353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:28.397651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e3ab6-c1fc-4ee5-a298-7fc47c660353 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:28.397933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e3ab6-c1fc-4ee5-a298-7fc47c660353 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:28.398209 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e3ab6-c1fc-4ee5-a298-7fc47c660353 None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:28.401050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e3ab6-c1fc-4ee5-a298-7fc47c660353 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:28.401050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e3ab6-c1fc-4ee5-a298-7fc47c660353 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:28.502466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 72/141] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:00:28 2022] POST /identity/v3/users => generated 283 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:00:29.701417 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278d55d-2566-4162-a19b-104786486474 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:29.701939 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278d55d-2566-4162-a19b-104786486474 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:29.702018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278d55d-2566-4162-a19b-104786486474 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:29.703444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 70/142] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:29.709020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3058323-4d37-414e-a11c-44fca98ca7aa None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:29.709439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3058323-4d37-414e-a11c-44fca98ca7aa None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:29.709627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3058323-4d37-414e-a11c-44fca98ca7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:29.733673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-b3058323-4d37-414e-a11c-44fca98ca7aa None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:29.734896 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b3058323-4d37-414e-a11c-44fca98ca7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:29.814807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 73/143] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:29.841698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e361c-e6ba-478b-a441-41fc06030156 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:29.841972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e361c-e6ba-478b-a441-41fc06030156 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:29.842267 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e361c-e6ba-478b-a441-41fc06030156 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:29.843754 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 71/144] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:29.849106 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09963064-b744-4594-93ba-dfb9d9f67d95 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:29.849404 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09963064-b744-4594-93ba-dfb9d9f67d95 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:29.849686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09963064-b744-4594-93ba-dfb9d9f67d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:29.868663 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-09963064-b744-4594-93ba-dfb9d9f67d95 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:29.869895 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-09963064-b744-4594-93ba-dfb9d9f67d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:29.940852 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 74/145] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:30.063787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dc9b7-6333-4c75-a038-07d3b2755865 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:30.064091 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dc9b7-6333-4c75-a038-07d3b2755865 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:30.064265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dc9b7-6333-4c75-a038-07d3b2755865 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:30.065701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 72/146] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:30.071636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe1fad-0900-4434-99ac-2096e45265f8 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:30.071934 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe1fad-0900-4434-99ac-2096e45265f8 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:30.072213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe1fad-0900-4434-99ac-2096e45265f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:30.094957 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-40fe1fad-0900-4434-99ac-2096e45265f8 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:30.096145 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-40fe1fad-0900-4434-99ac-2096e45265f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:30.145565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:30.173166 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 75/147] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:30.366994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dc9b7-6333-4c75-a038-07d3b2755865 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:30.429315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe1fad-0900-4434-99ac-2096e45265f8 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:30.481855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:30.526657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dc9b7-6333-4c75-a038-07d3b2755865 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:30.568253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe1fad-0900-4434-99ac-2096e45265f8 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:30.602908 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536ffad-fba1-4323-aa9d-f70238003ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:30.603658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536ffad-fba1-4323-aa9d-f70238003ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:30.603956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536ffad-fba1-4323-aa9d-f70238003ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:30.604234 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536ffad-fba1-4323-aa9d-f70238003ff5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:30.615941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab8bd73-3b7b-485e-a4b8-aa651f7c9da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:30.616585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8bd73-3b7b-485e-a4b8-aa651f7c9da6 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:30.616895 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8bd73-3b7b-485e-a4b8-aa651f7c9da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:30.617225 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8bd73-3b7b-485e-a4b8-aa651f7c9da6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:30.630706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab8bd73-3b7b-485e-a4b8-aa651f7c9da6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:30.634303 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536ffad-fba1-4323-aa9d-f70238003ff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-aab8bd73-3b7b-485e-a4b8-aa651f7c9da6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:30.636021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:30.636869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:30.637637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:30.637637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:30.637637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:30.637637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:30.637637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:30.637637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:30.637637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:30.637637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:30.637637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:30.637637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:30.637637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:30.637637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:30.637637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 76/148] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:30.638711 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd73124-629a-4716-89a0-370967f4f858 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:30.638975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd73124-629a-4716-89a0-370967f4f858 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-7536ffad-fba1-4323-aa9d-f70238003ff5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:30.639436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:30.640237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:30.642012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:30.642012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:30.642012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:30.642012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:30.642012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:30.642012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:30.642012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:30.642012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:30.642012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:30.642012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:30.642012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:30.642012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:30.643454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd73124-629a-4716-89a0-370967f4f858 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:30.643589 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 73/149] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:00:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:30.643719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 77/150] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:30.646531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536ffad-fba1-4323-aa9d-f70238003ff5 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:30.646711 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd73124-629a-4716-89a0-370967f4f858 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:30.692972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:30.771520 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd73124-629a-4716-89a0-370967f4f858 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:30.815615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938f681-646e-4200-8a75-00a6160ff0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:30.816483 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938f681-646e-4200-8a75-00a6160ff0ca None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:30.817200 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938f681-646e-4200-8a75-00a6160ff0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:30.818484 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938f681-646e-4200-8a75-00a6160ff0ca None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:30.819512 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938f681-646e-4200-8a75-00a6160ff0ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:30.821338 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938f681-646e-4200-8a75-00a6160ff0ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:30.822969 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938f681-646e-4200-8a75-00a6160ff0ca None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:30.839766 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 78/151] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:30.841811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938f681-646e-4200-8a75-00a6160ff0ca None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:30.857689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536ffad-fba1-4323-aa9d-f70238003ff5 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:30.928701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37de8cb-5cf8-474c-80fe-eb938aa642a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:30.929687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de8cb-5cf8-474c-80fe-eb938aa642a7 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:30.930682 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de8cb-5cf8-474c-80fe-eb938aa642a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:30.931168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de8cb-5cf8-474c-80fe-eb938aa642a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:30.954982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37de8cb-5cf8-474c-80fe-eb938aa642a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-b37de8cb-5cf8-474c-80fe-eb938aa642a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:30.960521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:30.961966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:30.962725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:30.962725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:30.962725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:30.962725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:30.962725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:30.962725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:30.962725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:30.962725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:30.962725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:30.962725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:30.962725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:30.962725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:30.965452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 74/152] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:00:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:30.966857 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b26f2e-f900-40d5-9584-399b4958ee54 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:30.967349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b26f2e-f900-40d5-9584-399b4958ee54 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:30.967820 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b26f2e-f900-40d5-9584-399b4958ee54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:30.980019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:30.995372 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-62b26f2e-f900-40d5-9584-399b4958ee54 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:30.997093 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-62b26f2e-f900-40d5-9584-399b4958ee54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:31.065017 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:31.069290 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938f681-646e-4200-8a75-00a6160ff0ca None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:31.144379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feece6f4-4903-4568-8c86-331025493b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:31.145033 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feece6f4-4903-4568-8c86-331025493b0d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:31.145316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feece6f4-4903-4568-8c86-331025493b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:31.145582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feece6f4-4903-4568-8c86-331025493b0d None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:31.146252 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feece6f4-4903-4568-8c86-331025493b0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:31.146612 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feece6f4-4903-4568-8c86-331025493b0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:31.147877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 75/153] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:31.147980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feece6f4-4903-4568-8c86-331025493b0d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:31.149624 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b26f2e-f900-40d5-9584-399b4958ee54 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:31.168670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 79/154] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:00:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:31.168670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feece6f4-4903-4568-8c86-331025493b0d None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:31.330759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b26f2e-f900-40d5-9584-399b4958ee54 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:31.384542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feece6f4-4903-4568-8c86-331025493b0d None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:31.397591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:31.413556 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d4ebe-dda5-4784-b534-ca21759c0803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:31.415078 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4ebe-dda5-4784-b534-ca21759c0803 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:31.417236 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4ebe-dda5-4784-b534-ca21759c0803 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:31.417236 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4ebe-dda5-4784-b534-ca21759c0803 None admin] PATH_INFO: `/v3/users/aaf97666c16d4c7684bda4f48b7cd610` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:31.418091 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d4ebe-dda5-4784-b534-ca21759c0803 None admin] RBAC: Authorizing `identity:get_user(user_id=aaf97666c16d4c7684bda4f48b7cd610)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:31.419735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d4ebe-dda5-4784-b534-ca21759c0803 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:31.421612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 76/155] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:31 2022] GET /identity/v3/users/aaf97666c16d4c7684bda4f48b7cd610 => generated 289 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:31.422593 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d4ebe-dda5-4784-b534-ca21759c0803 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:31.473392 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cced76-0d9a-4594-9274-2e5ae6e9fc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:31.474603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cced76-0d9a-4594-9274-2e5ae6e9fc34 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:31.474603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cced76-0d9a-4594-9274-2e5ae6e9fc34 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:31.474603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cced76-0d9a-4594-9274-2e5ae6e9fc34 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:31.475180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cced76-0d9a-4594-9274-2e5ae6e9fc34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:31.475485 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cced76-0d9a-4594-9274-2e5ae6e9fc34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:31.477406 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cced76-0d9a-4594-9274-2e5ae6e9fc34 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:31.491678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 80/156] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:31.493169 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370972d3-3d80-4b15-8f47-aa4ca97efe13 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:31.493461 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370972d3-3d80-4b15-8f47-aa4ca97efe13 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:31.493746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370972d3-3d80-4b15-8f47-aa4ca97efe13 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:31.494944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 81/157] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:31.497092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370972d3-3d80-4b15-8f47-aa4ca97efe13 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:31.551341 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:31.604137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d4ebe-dda5-4784-b534-ca21759c0803 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:31.657274 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370972d3-3d80-4b15-8f47-aa4ca97efe13 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:31.660514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c53d7f-630f-4675-a54e-dd3eab41c298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:31.661112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c53d7f-630f-4675-a54e-dd3eab41c298 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:31.661398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c53d7f-630f-4675-a54e-dd3eab41c298 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:31.661679 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c53d7f-630f-4675-a54e-dd3eab41c298 None admin] PATH_INFO: `/v3/users/41b76a7e18b64d968720b907624136d4` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:31.665691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c53d7f-630f-4675-a54e-dd3eab41c298 None admin] RBAC: Authorizing `identity:get_user(user_id=41b76a7e18b64d968720b907624136d4)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:31.667409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c53d7f-630f-4675-a54e-dd3eab41c298 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:31.669380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 77/158] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:31 2022] GET /identity/v3/users/41b76a7e18b64d968720b907624136d4 => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:31.670667 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019befa9-7c71-4dc1-a646-6189979150f6 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:31.670781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019befa9-7c71-4dc1-a646-6189979150f6 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:31.671035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019befa9-7c71-4dc1-a646-6189979150f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:31.697990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-019befa9-7c71-4dc1-a646-6189979150f6 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:31.699180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-019befa9-7c71-4dc1-a646-6189979150f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:31.713348 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887f588-0d50-4a9f-8c38-46c80f96c017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:31.714224 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887f588-0d50-4a9f-8c38-46c80f96c017 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:31.714651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887f588-0d50-4a9f-8c38-46c80f96c017 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:31.715065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887f588-0d50-4a9f-8c38-46c80f96c017 None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:31.716553 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887f588-0d50-4a9f-8c38-46c80f96c017 None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:31.718170 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887f588-0d50-4a9f-8c38-46c80f96c017 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:31.719987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 82/159] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:31 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:31.720926 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887f588-0d50-4a9f-8c38-46c80f96c017 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:31.817828 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:31.824830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 78/160] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:31.825660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019befa9-7c71-4dc1-a646-6189979150f6 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:31.895872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:31.915917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887f588-0d50-4a9f-8c38-46c80f96c017 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:31.979474 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe28cc-e6d8-4fd7-b08d-e0a8e6e6bff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:31.980107 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe28cc-e6d8-4fd7-b08d-e0a8e6e6bff7 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:31.980383 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe28cc-e6d8-4fd7-b08d-e0a8e6e6bff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:31.980669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe28cc-e6d8-4fd7-b08d-e0a8e6e6bff7 None admin] PATH_INFO: `/v3/users/1b637f155b514b58ab1c4d9c02381dc6` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:31.981833 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe28cc-e6d8-4fd7-b08d-e0a8e6e6bff7 None admin] RBAC: Authorizing `identity:get_user(user_id=1b637f155b514b58ab1c4d9c02381dc6)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:31.984009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe28cc-e6d8-4fd7-b08d-e0a8e6e6bff7 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:31.987551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 83/161] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:31 2022] GET /identity/v3/users/1b637f155b514b58ab1c4d9c02381dc6 => generated 247 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:31.988326 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe28cc-e6d8-4fd7-b08d-e0a8e6e6bff7 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:32.026673 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019befa9-7c71-4dc1-a646-6189979150f6 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:32.095264 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56e1a1-119b-461c-bb66-68b68a7ba4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:32.095917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56e1a1-119b-461c-bb66-68b68a7ba4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:32.096185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56e1a1-119b-461c-bb66-68b68a7ba4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:32.096456 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56e1a1-119b-461c-bb66-68b68a7ba4a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:32.120938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56e1a1-119b-461c-bb66-68b68a7ba4a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-eb56e1a1-119b-461c-bb66-68b68a7ba4a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:32.128446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:32.129483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:32.130239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:32.130239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:32.130239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:32.130239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:32.130239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:32.130239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:32.130239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:32.130239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:32.130239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:32.130239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:32.130239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:32.130239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:32.130239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 79/162] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:32.130239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56e1a1-119b-461c-bb66-68b68a7ba4a2 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:32.146308 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe28cc-e6d8-4fd7-b08d-e0a8e6e6bff7 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:32.193650 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90b155-bfed-431c-a663-ea341002d8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:32.194658 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90b155-bfed-431c-a663-ea341002d8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:32.195609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90b155-bfed-431c-a663-ea341002d8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:32.196568 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90b155-bfed-431c-a663-ea341002d8e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:32.230101 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:32.242094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:32.313545 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90b155-bfed-431c-a663-ea341002d8e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:32.315594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90b155-bfed-431c-a663-ea341002d8e2 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:32.330155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56e1a1-119b-461c-bb66-68b68a7ba4a2 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:32.366477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f9dd7-1502-4585-b32e-9b4bcba4352e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:32.367279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f9dd7-1502-4585-b32e-9b4bcba4352e None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:32.368121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f9dd7-1502-4585-b32e-9b4bcba4352e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:32.368497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f9dd7-1502-4585-b32e-9b4bcba4352e None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4/users/aaf97666c16d4c7684bda4f48b7cd610/roles/bf2de8c80c5e4be79e1ea7adc3cbfc56` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:32.370531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f9dd7-1502-4585-b32e-9b4bcba4352e None admin] RBAC: Authorizing `identity:create_grant(project_id=4fe97b975d834d83a6733a0b2591bba4, user_id=aaf97666c16d4c7684bda4f48b7cd610, role_id=bf2de8c80c5e4be79e1ea7adc3cbfc56)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:32.371991 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f9dd7-1502-4585-b32e-9b4bcba4352e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:32.436009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 84/163] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:32.436877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90b155-bfed-431c-a663-ea341002d8e2 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:32.444391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 80/164] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:00:32 2022] PUT /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4/users/aaf97666c16d4c7684bda4f48b7cd610/roles/bf2de8c80c5e4be79e1ea7adc3cbfc56 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:00:32.448842 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f9dd7-1502-4585-b32e-9b4bcba4352e None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:32.588977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f9dd7-1502-4585-b32e-9b4bcba4352e None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:32.595436 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:32.613544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:32.646466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3931d-8aba-45c3-ba31-95d8894eaba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:32.647093 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3931d-8aba-45c3-ba31-95d8894eaba0 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:32.647377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3931d-8aba-45c3-ba31-95d8894eaba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:32.647653 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3931d-8aba-45c3-ba31-95d8894eaba0 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:32.648336 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3931d-8aba-45c3-ba31-95d8894eaba0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:32.648713 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3931d-8aba-45c3-ba31-95d8894eaba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:32.655614 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3931d-8aba-45c3-ba31-95d8894eaba0 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:32.665523 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 81/165] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:32.666893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5c43a-44de-442c-8d58-4716b1bf0bd1 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:32.667214 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5c43a-44de-442c-8d58-4716b1bf0bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:32.667453 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5c43a-44de-442c-8d58-4716b1bf0bd1 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:32.669841 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 82/166] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:32.670651 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5c43a-44de-442c-8d58-4716b1bf0bd1 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:32.678025 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90b155-bfed-431c-a663-ea341002d8e2 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:32.727426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4c267-433d-48bb-b57f-a6c6ee93b310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:32.728368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4c267-433d-48bb-b57f-a6c6ee93b310 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:32.729008 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4c267-433d-48bb-b57f-a6c6ee93b310 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:32.729484 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4c267-433d-48bb-b57f-a6c6ee93b310 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:32.804378 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4c267-433d-48bb-b57f-a6c6ee93b310 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:32.806888 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4c267-433d-48bb-b57f-a6c6ee93b310 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:32.873331 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5c43a-44de-442c-8d58-4716b1bf0bd1 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:32.878047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:32.915657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 85/167] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:32.916656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4c267-433d-48bb-b57f-a6c6ee93b310 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:32.939681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678bf80-0465-4a1f-a58f-324e0de89293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:32.940415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678bf80-0465-4a1f-a58f-324e0de89293 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:32.940727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678bf80-0465-4a1f-a58f-324e0de89293 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:32.941007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678bf80-0465-4a1f-a58f-324e0de89293 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:32.953154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678bf80-0465-4a1f-a58f-324e0de89293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-a678bf80-0465-4a1f-a58f-324e0de89293 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:32.958586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:32.959483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:32.960262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:32.960262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:32.960262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:32.960262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:32.960262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:32.960262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:32.960262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:32.960262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:32.960262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:32.960262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:32.960262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:32.960262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:32.960262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 83/168] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:32.960956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678bf80-0465-4a1f-a58f-324e0de89293 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:33.006323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:33.032797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4c267-433d-48bb-b57f-a6c6ee93b310 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:33.070756 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678bf80-0465-4a1f-a58f-324e0de89293 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:33.072153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce21f8-0006-47f5-a08f-38c0f259a8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:33.073011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce21f8-0006-47f5-a08f-38c0f259a8ed None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:33.073361 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce21f8-0006-47f5-a08f-38c0f259a8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:33.073703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce21f8-0006-47f5-a08f-38c0f259a8ed None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:33.074919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce21f8-0006-47f5-a08f-38c0f259a8ed None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:33.076504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce21f8-0006-47f5-a08f-38c0f259a8ed None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:33.083297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 86/169] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:32 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:33.084798 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77fe09-b839-4b3c-b1f1-4abf524d3f3e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:33.085161 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77fe09-b839-4b3c-b1f1-4abf524d3f3e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:33.085507 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77fe09-b839-4b3c-b1f1-4abf524d3f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:33.106299 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-3a77fe09-b839-4b3c-b1f1-4abf524d3f3e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:33.106583 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a4d65-0def-4389-82ab-e8eacd5c7fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:33.107210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a4d65-0def-4389-82ab-e8eacd5c7fcf None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:33.107493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a4d65-0def-4389-82ab-e8eacd5c7fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:33.107767 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a4d65-0def-4389-82ab-e8eacd5c7fcf None admin] PATH_INFO: `/v3/users/41b76a7e18b64d968720b907624136d4` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:33.107915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-3a77fe09-b839-4b3c-b1f1-4abf524d3f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:33.108956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a4d65-0def-4389-82ab-e8eacd5c7fcf None admin] RBAC: Authorizing `identity:get_user(user_id=41b76a7e18b64d968720b907624136d4)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:33.110536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a4d65-0def-4389-82ab-e8eacd5c7fcf None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:33.112240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 84/170] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:32 2022] GET /identity/v3/users/41b76a7e18b64d968720b907624136d4 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:33.113185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a4d65-0def-4389-82ab-e8eacd5c7fcf None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:33.137238 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:33.191779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 87/171] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:33.201970 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e8008-3bf5-4709-ba93-58cf28864428 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:33.202265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e8008-3bf5-4709-ba93-58cf28864428 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:33.202544 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e8008-3bf5-4709-ba93-58cf28864428 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:33.204178 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 88/172] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:33 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:33.204915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e8008-3bf5-4709-ba93-58cf28864428 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:33.239695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a4d65-0def-4389-82ab-e8eacd5c7fcf None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:33.250837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:33.282044 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4def86-1113-4afd-b26a-9f1f8c438d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:33.283209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4def86-1113-4afd-b26a-9f1f8c438d2d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:33.284042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4def86-1113-4afd-b26a-9f1f8c438d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:33.284487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4def86-1113-4afd-b26a-9f1f8c438d2d None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:33.285971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4def86-1113-4afd-b26a-9f1f8c438d2d None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:33.287655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4def86-1113-4afd-b26a-9f1f8c438d2d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:33.289536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 85/173] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:33 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:33.290893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4def86-1113-4afd-b26a-9f1f8c438d2d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:33.361494 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e8008-3bf5-4709-ba93-58cf28864428 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:33.398486 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ac76a-83d5-429a-9d33-20aa2ece77bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:33.399032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ac76a-83d5-429a-9d33-20aa2ece77bb None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:33.399321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ac76a-83d5-429a-9d33-20aa2ece77bb None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:33.399574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ac76a-83d5-429a-9d33-20aa2ece77bb None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:33.400237 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ac76a-83d5-429a-9d33-20aa2ece77bb None admin] RBAC: Adding query filter params (name=member) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:33.400542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ac76a-83d5-429a-9d33-20aa2ece77bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:33.401949 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ac76a-83d5-429a-9d33-20aa2ece77bb None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:33.410210 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 89/174] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:33.411079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ac76a-83d5-429a-9d33-20aa2ece77bb None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:33.438677 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4def86-1113-4afd-b26a-9f1f8c438d2d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:33.439565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:33.486905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07744dd5-e9a1-4c7f-b5f3-e608aee17891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:33.488096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07744dd5-e9a1-4c7f-b5f3-e608aee17891 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:33.488996 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07744dd5-e9a1-4c7f-b5f3-e608aee17891 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:33.489503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07744dd5-e9a1-4c7f-b5f3-e608aee17891 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:33.491718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07744dd5-e9a1-4c7f-b5f3-e608aee17891 None admin] RBAC: Adding query filter params (role.id=b5e411fc96434df69c6cc1df951b3e6f, scope.project.id=4fe97b975d834d83a6733a0b2591bba4, user.id=41b76a7e18b64d968720b907624136d4) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:33.492242 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07744dd5-e9a1-4c7f-b5f3-e608aee17891 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:33.495167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07744dd5-e9a1-4c7f-b5f3-e608aee17891 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:33.502771 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 86/175] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:33 2022] GET /identity/v3/role_assignments?user.id=41b76a7e18b64d968720b907624136d4&scope.project.id=4fe97b975d834d83a6733a0b2591bba4&role.id=b5e411fc96434df69c6cc1df951b3e6f => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:33.504711 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef0858-0a1a-41dd-a5d1-70486c6205ee None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:33.505206 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef0858-0a1a-41dd-a5d1-70486c6205ee None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:33.505668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef0858-0a1a-41dd-a5d1-70486c6205ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:33.533177 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ac76a-83d5-429a-9d33-20aa2ece77bb None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:33.534017 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-d4ef0858-0a1a-41dd-a5d1-70486c6205ee None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:33.536766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-d4ef0858-0a1a-41dd-a5d1-70486c6205ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:33.576277 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd3ef2-fe85-4983-946e-3765489ae183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:33.576927 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd3ef2-fe85-4983-946e-3765489ae183 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:33.577206 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd3ef2-fe85-4983-946e-3765489ae183 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:33.577502 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd3ef2-fe85-4983-946e-3765489ae183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:33.598330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:33.645021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 87/176] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:33.646242 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef0858-0a1a-41dd-a5d1-70486c6205ee None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:33.673706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd3ef2-fe85-4983-946e-3765489ae183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:33.675579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd3ef2-fe85-4983-946e-3765489ae183 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:33.697218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:33.785048 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef0858-0a1a-41dd-a5d1-70486c6205ee None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:33.805980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 90/177] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:33.808480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3d86f-3f0f-4c4a-a108-bea7a7c0f3cb None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:33.813392 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3d86f-3f0f-4c4a-a108-bea7a7c0f3cb None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:33.813392 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3d86f-3f0f-4c4a-a108-bea7a7c0f3cb None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:33.817030 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 91/178] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:33 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:33.818653 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3d86f-3f0f-4c4a-a108-bea7a7c0f3cb None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:33.842889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fbdb1-0030-4b61-bf5d-90c8016040bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:33.843564 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fbdb1-0030-4b61-bf5d-90c8016040bc None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:33.843848 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fbdb1-0030-4b61-bf5d-90c8016040bc None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:33.844123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fbdb1-0030-4b61-bf5d-90c8016040bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:33.844956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fbdb1-0030-4b61-bf5d-90c8016040bc None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=4fe97b975d834d83a6733a0b2591bba4, user.id=1b637f155b514b58ab1c4d9c02381dc6) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:33.845266 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fbdb1-0030-4b61-bf5d-90c8016040bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:33.846977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fbdb1-0030-4b61-bf5d-90c8016040bc None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:33.853401 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 88/179] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:33 2022] GET /identity/v3/role_assignments?user.id=1b637f155b514b58ab1c4d9c02381dc6&scope.project.id=4fe97b975d834d83a6733a0b2591bba4&role.id=906e3a8a6c7540a38f629f2567133347 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:33.854320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fbdb1-0030-4b61-bf5d-90c8016040bc None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:33.894663 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:33.950204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3d86f-3f0f-4c4a-a108-bea7a7c0f3cb None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:33.969856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fbdb1-0030-4b61-bf5d-90c8016040bc None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:33.973331 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:33.994761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91c026-737d-477f-aba7-a46054da10bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:33.995511 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91c026-737d-477f-aba7-a46054da10bf None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:33.995788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91c026-737d-477f-aba7-a46054da10bf None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:33.996057 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91c026-737d-477f-aba7-a46054da10bf None admin] PATH_INFO: `/v3/users/41b76a7e18b64d968720b907624136d4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:33.997252 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91c026-737d-477f-aba7-a46054da10bf None admin] RBAC: Authorizing `identity:get_user(user_id=41b76a7e18b64d968720b907624136d4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:33.998829 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91c026-737d-477f-aba7-a46054da10bf None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:34.000834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 92/180] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:33 2022] GET /identity/v3/users/41b76a7e18b64d968720b907624136d4 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:34.001654 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91c026-737d-477f-aba7-a46054da10bf None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:34.015514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236096d7-c0ea-47c8-a1f8-60ec4282a6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:34.016310 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236096d7-c0ea-47c8-a1f8-60ec4282a6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:34.016735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236096d7-c0ea-47c8-a1f8-60ec4282a6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:34.017086 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236096d7-c0ea-47c8-a1f8-60ec4282a6a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:34.030308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236096d7-c0ea-47c8-a1f8-60ec4282a6a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-236096d7-c0ea-47c8-a1f8-60ec4282a6a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:34.037330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:34.038399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:34.039872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:34.039872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:34.039872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:34.039872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:34.039872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:34.039872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:34.039872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:34.039872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:34.039872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:34.039872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:34.039872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:34.039872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:34.039872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 89/181] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:00:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:34.041035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccd64e-c4b1-4d93-b94d-530d57705c6a None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:34.041035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccd64e-c4b1-4d93-b94d-530d57705c6a None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:34.041035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccd64e-c4b1-4d93-b94d-530d57705c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:34.062552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-6fccd64e-c4b1-4d93-b94d-530d57705c6a None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:34.063874 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-6fccd64e-c4b1-4d93-b94d-530d57705c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:34.131908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91c026-737d-477f-aba7-a46054da10bf None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:34.157291 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 90/182] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:34.159292 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccd64e-c4b1-4d93-b94d-530d57705c6a None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:34.180934 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e77c8-2b62-4b41-b5c9-a3e9ea21d40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:34.182472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e77c8-2b62-4b41-b5c9-a3e9ea21d40b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:34.182578 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e77c8-2b62-4b41-b5c9-a3e9ea21d40b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:34.182874 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e77c8-2b62-4b41-b5c9-a3e9ea21d40b None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:34.184109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e77c8-2b62-4b41-b5c9-a3e9ea21d40b None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:34.185714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e77c8-2b62-4b41-b5c9-a3e9ea21d40b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:34.188463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 93/183] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:34 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:34.189264 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e77c8-2b62-4b41-b5c9-a3e9ea21d40b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:34.196749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:34.281012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:34.302622 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccd64e-c4b1-4d93-b94d-530d57705c6a None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:34.331445 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e77c8-2b62-4b41-b5c9-a3e9ea21d40b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:34.349410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e89cf3-5fc5-4580-a4d9-ee69127ca068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:34.350398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e89cf3-5fc5-4580-a4d9-ee69127ca068 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:34.350875 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e89cf3-5fc5-4580-a4d9-ee69127ca068 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:34.352355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e89cf3-5fc5-4580-a4d9-ee69127ca068 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:34.364222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e89cf3-5fc5-4580-a4d9-ee69127ca068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-48e89cf3-5fc5-4580-a4d9-ee69127ca068 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:34.371763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:34.372615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:34.373402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:34.373402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:34.373402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:34.373402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:34.373402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:34.373402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:34.373402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:34.373402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:34.373402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:34.373402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:34.373402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:34.373402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:34.373988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114efdcb-41c1-4de6-93dc-a7d34e476c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:34.373988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114efdcb-41c1-4de6-93dc-a7d34e476c0f None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:34.373988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114efdcb-41c1-4de6-93dc-a7d34e476c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:34.373988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114efdcb-41c1-4de6-93dc-a7d34e476c0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:34.375988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 91/184] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:00:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:34.379916 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e89cf3-5fc5-4580-a4d9-ee69127ca068 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:34.463371 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114efdcb-41c1-4de6-93dc-a7d34e476c0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:34.465029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114efdcb-41c1-4de6-93dc-a7d34e476c0f None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:34.487817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:34.503974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e89cf3-5fc5-4580-a4d9-ee69127ca068 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:34.529582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:34.540186 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1514e0f-769a-4370-8f0a-806661671e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:34.545330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1514e0f-769a-4370-8f0a-806661671e2a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:34.545330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1514e0f-769a-4370-8f0a-806661671e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:34.545330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1514e0f-769a-4370-8f0a-806661671e2a None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:34.545330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1514e0f-769a-4370-8f0a-806661671e2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:34.545330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1514e0f-769a-4370-8f0a-806661671e2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:34.545330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1514e0f-769a-4370-8f0a-806661671e2a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:34.552256 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 92/185] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:34.553069 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1514e0f-769a-4370-8f0a-806661671e2a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:34.580908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 94/186] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:34.581977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114efdcb-41c1-4de6-93dc-a7d34e476c0f None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:34.661719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1514e0f-769a-4370-8f0a-806661671e2a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:34.690007 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114efdcb-41c1-4de6-93dc-a7d34e476c0f None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:34.698563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156a325-580e-4ec0-bdf7-8a24cdbbad03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:34.699164 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156a325-580e-4ec0-bdf7-8a24cdbbad03 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:34.699447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156a325-580e-4ec0-bdf7-8a24cdbbad03 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:34.699719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156a325-580e-4ec0-bdf7-8a24cdbbad03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:34.700451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156a325-580e-4ec0-bdf7-8a24cdbbad03 None admin] RBAC: Adding query filter params (role.id=8b2ee9b11a934b67aa3fd68c497fbdf7, scope.project.id=4fe97b975d834d83a6733a0b2591bba4, user.id=41b76a7e18b64d968720b907624136d4) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:34.700792 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156a325-580e-4ec0-bdf7-8a24cdbbad03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:34.702246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156a325-580e-4ec0-bdf7-8a24cdbbad03 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:34.716253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 93/187] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:34 2022] GET /identity/v3/role_assignments?user.id=41b76a7e18b64d968720b907624136d4&scope.project.id=4fe97b975d834d83a6733a0b2591bba4&role.id=8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:34.716253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156a325-580e-4ec0-bdf7-8a24cdbbad03 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:34.723458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:34.746559 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a89ce-ceb1-4e41-a2a6-3ac81a4c8601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:34.747395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a89ce-ceb1-4e41-a2a6-3ac81a4c8601 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:34.747852 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a89ce-ceb1-4e41-a2a6-3ac81a4c8601 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:34.748133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a89ce-ceb1-4e41-a2a6-3ac81a4c8601 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:34.749537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a89ce-ceb1-4e41-a2a6-3ac81a4c8601 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:34.749855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a89ce-ceb1-4e41-a2a6-3ac81a4c8601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:34.751877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a89ce-ceb1-4e41-a2a6-3ac81a4c8601 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:34.765812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 95/188] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:00:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:34.767650 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a89ce-ceb1-4e41-a2a6-3ac81a4c8601 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:34.790886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:34.833224 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156a325-580e-4ec0-bdf7-8a24cdbbad03 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:34.875759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960209f7-309c-4c5e-aac7-0eab6f94c493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:34.876399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960209f7-309c-4c5e-aac7-0eab6f94c493 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:34.876724 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960209f7-309c-4c5e-aac7-0eab6f94c493 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:34.877007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960209f7-309c-4c5e-aac7-0eab6f94c493 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:34.878136 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960209f7-309c-4c5e-aac7-0eab6f94c493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:34.879604 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960209f7-309c-4c5e-aac7-0eab6f94c493 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:34.881444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 94/189] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:34.882222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960209f7-309c-4c5e-aac7-0eab6f94c493 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:34.897012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a89ce-ceb1-4e41-a2a6-3ac81a4c8601 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:34.941307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c87005-1008-47f2-854c-498e8a84ce8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:34.942174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c87005-1008-47f2-854c-498e8a84ce8a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:34.942600 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c87005-1008-47f2-854c-498e8a84ce8a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:34.943027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c87005-1008-47f2-854c-498e8a84ce8a None admin] PATH_INFO: `/v3/users/c9421b5ced544e1591c7ab5d6204d5f0` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:34.956589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c87005-1008-47f2-854c-498e8a84ce8a None admin] RBAC: Authorizing `identity:get_user(user_id=c9421b5ced544e1591c7ab5d6204d5f0)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:34.958075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c87005-1008-47f2-854c-498e8a84ce8a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:34.959802 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 96/190] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:34 2022] GET /identity/v3/users/c9421b5ced544e1591c7ab5d6204d5f0 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:34.961075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b0898-57e6-4691-a7c3-768a2cf207f2 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:34.961403 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b0898-57e6-4691-a7c3-768a2cf207f2 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:34.961777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b0898-57e6-4691-a7c3-768a2cf207f2 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:34.963108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 97/191] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:34.965391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b0898-57e6-4691-a7c3-768a2cf207f2 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:34.978188 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:34.998930 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960209f7-309c-4c5e-aac7-0eab6f94c493 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:35.033075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305af45-4a7e-48af-95cd-59212c98d27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:35.033724 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305af45-4a7e-48af-95cd-59212c98d27a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:35.034000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305af45-4a7e-48af-95cd-59212c98d27a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:35.034270 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305af45-4a7e-48af-95cd-59212c98d27a None admin] PATH_INFO: `/v3/projects/b27e60e99095427f972d99e4d3b16c9c` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:35.035369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2305af45-4a7e-48af-95cd-59212c98d27a None admin] RBAC: Authorizing `identity:get_project(project_id=b27e60e99095427f972d99e4d3b16c9c)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:35.036871 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2305af45-4a7e-48af-95cd-59212c98d27a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:35.038591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 95/192] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:34 2022] GET /identity/v3/projects/b27e60e99095427f972d99e4d3b16c9c => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:35.039414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305af45-4a7e-48af-95cd-59212c98d27a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:35.041604 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:35.068384 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b0898-57e6-4691-a7c3-768a2cf207f2 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:35.103218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409fd25-3c54-4795-90f0-9361aea73257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:35.103862 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409fd25-3c54-4795-90f0-9361aea73257 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:35.104122 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409fd25-3c54-4795-90f0-9361aea73257 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:35.104397 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409fd25-3c54-4795-90f0-9361aea73257 None admin] PATH_INFO: `/v3/users/c9421b5ced544e1591c7ab5d6204d5f0` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:35.105560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409fd25-3c54-4795-90f0-9361aea73257 None admin] RBAC: Authorizing `identity:get_user(user_id=c9421b5ced544e1591c7ab5d6204d5f0)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:35.107045 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409fd25-3c54-4795-90f0-9361aea73257 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:35.108615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 98/193] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:34 2022] GET /identity/v3/users/c9421b5ced544e1591c7ab5d6204d5f0 => generated 283 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:35.109437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409fd25-3c54-4795-90f0-9361aea73257 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:35.145871 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305af45-4a7e-48af-95cd-59212c98d27a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:35.178024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235dad2b-64c0-4acf-8abc-3a99c527dd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:35.178631 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235dad2b-64c0-4acf-8abc-3a99c527dd2b None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:35.178891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235dad2b-64c0-4acf-8abc-3a99c527dd2b None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:35.179165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235dad2b-64c0-4acf-8abc-3a99c527dd2b None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:35.179839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235dad2b-64c0-4acf-8abc-3a99c527dd2b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:35.181305 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235dad2b-64c0-4acf-8abc-3a99c527dd2b None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:35.208507 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409fd25-3c54-4795-90f0-9361aea73257 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:35.242849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d3143-41dc-414b-b66a-e1b80d1e366e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:35.243531 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d3143-41dc-414b-b66a-e1b80d1e366e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:35.243809 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d3143-41dc-414b-b66a-e1b80d1e366e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:35.244076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d3143-41dc-414b-b66a-e1b80d1e366e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:35.255465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:35.266681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 96/194] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:00:35 2022] POST /identity/v3/users => generated 297 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:00:35.274318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f21a2-c4e9-4947-b9f9-adf396baccf3 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:35.274601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f21a2-c4e9-4947-b9f9-adf396baccf3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:35.275212 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f21a2-c4e9-4947-b9f9-adf396baccf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:35.281812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:35.294448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-1d3f21a2-c4e9-4947-b9f9-adf396baccf3 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:35.295666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-1d3f21a2-c4e9-4947-b9f9-adf396baccf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:35.317142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d3143-41dc-414b-b66a-e1b80d1e366e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:35.318834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d3143-41dc-414b-b66a-e1b80d1e366e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:35.384670 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 97/195] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:35.384670 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f21a2-c4e9-4947-b9f9-adf396baccf3 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:35.461628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 99/196] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:35.472838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d3143-41dc-414b-b66a-e1b80d1e366e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:35.520165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f21a2-c4e9-4947-b9f9-adf396baccf3 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:35.537969 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:35.557029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:35.562277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efa022-f9a6-4ec8-85ca-ceb643454fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:35.562566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efa022-f9a6-4ec8-85ca-ceb643454fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:35.563806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efa022-f9a6-4ec8-85ca-ceb643454fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:35.563806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efa022-f9a6-4ec8-85ca-ceb643454fb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:35.563963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52efa022-f9a6-4ec8-85ca-ceb643454fb4 None admin] RBAC: Adding query filter params (role.id=8b2ee9b11a934b67aa3fd68c497fbdf7, scope.project.id=b27e60e99095427f972d99e4d3b16c9c, user.id=41b76a7e18b64d968720b907624136d4) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:35.564154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52efa022-f9a6-4ec8-85ca-ceb643454fb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:35.566384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52efa022-f9a6-4ec8-85ca-ceb643454fb4 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:35.573673 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 98/197] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:35 2022] GET /identity/v3/role_assignments?user.id=41b76a7e18b64d968720b907624136d4&scope.project.id=b27e60e99095427f972d99e4d3b16c9c&role.id=8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:35.574864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efa022-f9a6-4ec8-85ca-ceb643454fb4 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:35.618137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d3143-41dc-414b-b66a-e1b80d1e366e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:35.658739 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348b7ae-5fc2-4222-8393-eb7302c95707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:35.660322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348b7ae-5fc2-4222-8393-eb7302c95707 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:35.660845 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348b7ae-5fc2-4222-8393-eb7302c95707 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:35.661292 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348b7ae-5fc2-4222-8393-eb7302c95707 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:35.721993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efa022-f9a6-4ec8-85ca-ceb643454fb4 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:35.753368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348b7ae-5fc2-4222-8393-eb7302c95707 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:35.755140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348b7ae-5fc2-4222-8393-eb7302c95707 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:35.768553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2caf60-3da3-4f2c-9d30-95dbe60b7683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:35.769260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2caf60-3da3-4f2c-9d30-95dbe60b7683 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:35.769541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2caf60-3da3-4f2c-9d30-95dbe60b7683 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:35.769812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2caf60-3da3-4f2c-9d30-95dbe60b7683 None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:35.771325 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2caf60-3da3-4f2c-9d30-95dbe60b7683 None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:35.772434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2caf60-3da3-4f2c-9d30-95dbe60b7683 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:35.774332 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 99/198] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:35 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:35.775114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2caf60-3da3-4f2c-9d30-95dbe60b7683 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:35.812351 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:35.866659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:35.884445 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 100/199] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:35.885340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348b7ae-5fc2-4222-8393-eb7302c95707 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:35.927438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2caf60-3da3-4f2c-9d30-95dbe60b7683 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:35.987823 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ca05a-6f16-474e-9c64-5cd3dd05d875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:35.988574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ca05a-6f16-474e-9c64-5cd3dd05d875 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:35.989576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ca05a-6f16-474e-9c64-5cd3dd05d875 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:35.989576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ca05a-6f16-474e-9c64-5cd3dd05d875 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:36.003901 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ca05a-6f16-474e-9c64-5cd3dd05d875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-d43ca05a-6f16-474e-9c64-5cd3dd05d875 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:36.010245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:36.011111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:36.011868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:36.011868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:36.011868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:36.011868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:36.011868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:36.011868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:36.011868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:36.011868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:36.011868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:36.011868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:36.011868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:36.011868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:36.011868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 100/200] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:36.012508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ca05a-6f16-474e-9c64-5cd3dd05d875 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:36.027259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348b7ae-5fc2-4222-8393-eb7302c95707 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:36.065313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aef881-e472-4460-8f55-b61a5d5396ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:36.066356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aef881-e472-4460-8f55-b61a5d5396ad None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:36.066908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aef881-e472-4460-8f55-b61a5d5396ad None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:36.067376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aef881-e472-4460-8f55-b61a5d5396ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:36.068298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aef881-e472-4460-8f55-b61a5d5396ad None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=ad5d2677b95b428a93e499f4ced1b86b, user.id=c9421b5ced544e1591c7ab5d6204d5f0) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:36.068855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aef881-e472-4460-8f55-b61a5d5396ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:36.070982 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aef881-e472-4460-8f55-b61a5d5396ad None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:36.077722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 101/201] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:35 2022] GET /identity/v3/role_assignments?user.id=c9421b5ced544e1591c7ab5d6204d5f0&scope.project.id=ad5d2677b95b428a93e499f4ced1b86b&role.id=906e3a8a6c7540a38f629f2567133347 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:36.084683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aef881-e472-4460-8f55-b61a5d5396ad None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:36.095499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:36.128767 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ca05a-6f16-474e-9c64-5cd3dd05d875 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:36.160111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:36.169111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b22fcc-73db-4302-ad67-a0d2fd3ee4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:36.170095 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b22fcc-73db-4302-ad67-a0d2fd3ee4a7 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:36.170543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b22fcc-73db-4302-ad67-a0d2fd3ee4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:36.173168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b22fcc-73db-4302-ad67-a0d2fd3ee4a7 None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:36.173616 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b22fcc-73db-4302-ad67-a0d2fd3ee4a7 None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:36.176032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b22fcc-73db-4302-ad67-a0d2fd3ee4a7 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:36.179490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 101/202] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:36 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:36.186697 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b22fcc-73db-4302-ad67-a0d2fd3ee4a7 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:36.281423 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aef881-e472-4460-8f55-b61a5d5396ad None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:36.329880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f42fb-e29b-4da8-99e3-a954735560d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:36.330251 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f42fb-e29b-4da8-99e3-a954735560d3 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:36.330509 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f42fb-e29b-4da8-99e3-a954735560d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:36.330780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f42fb-e29b-4da8-99e3-a954735560d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:36.332560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f42fb-e29b-4da8-99e3-a954735560d3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:36.332560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f42fb-e29b-4da8-99e3-a954735560d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:36.333133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f42fb-e29b-4da8-99e3-a954735560d3 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:36.344777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 102/203] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:36.350063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f42fb-e29b-4da8-99e3-a954735560d3 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:36.367850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b22fcc-73db-4302-ad67-a0d2fd3ee4a7 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:36.446464 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c483d1a-29bb-45a6-922d-5b0260e85f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:36.447445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c483d1a-29bb-45a6-922d-5b0260e85f7a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:36.447924 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c483d1a-29bb-45a6-922d-5b0260e85f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:36.448409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c483d1a-29bb-45a6-922d-5b0260e85f7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:36.449383 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c483d1a-29bb-45a6-922d-5b0260e85f7a None admin] RBAC: Adding query filter params (role.id=b5e411fc96434df69c6cc1df951b3e6f, scope.project.id=ad5d2677b95b428a93e499f4ced1b86b, user.id=c9421b5ced544e1591c7ab5d6204d5f0) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:36.449920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c483d1a-29bb-45a6-922d-5b0260e85f7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:36.451551 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c483d1a-29bb-45a6-922d-5b0260e85f7a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:36.457470 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 102/204] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:36 2022] GET /identity/v3/role_assignments?user.id=c9421b5ced544e1591c7ab5d6204d5f0&scope.project.id=ad5d2677b95b428a93e499f4ced1b86b&role.id=b5e411fc96434df69c6cc1df951b3e6f => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:36.468681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:36.503220 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f42fb-e29b-4da8-99e3-a954735560d3 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:36.541619 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd37bd-6ec9-4286-8263-6e74e580c1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:36.542331 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd37bd-6ec9-4286-8263-6e74e580c1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:36.542619 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd37bd-6ec9-4286-8263-6e74e580c1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:36.542890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd37bd-6ec9-4286-8263-6e74e580c1f4 None admin] PATH_INFO: `/v3/users/aaf97666c16d4c7684bda4f48b7cd610` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:36.544138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd37bd-6ec9-4286-8263-6e74e580c1f4 None admin] RBAC: Authorizing `identity:get_user(user_id=aaf97666c16d4c7684bda4f48b7cd610)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:36.545763 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd37bd-6ec9-4286-8263-6e74e580c1f4 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:36.547545 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 103/205] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:36 2022] GET /identity/v3/users/aaf97666c16d4c7684bda4f48b7cd610 => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:36.553294 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c483d1a-29bb-45a6-922d-5b0260e85f7a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:36.615699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:36.665341 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c483d1a-29bb-45a6-922d-5b0260e85f7a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:36.701375 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11527a5a-e644-4b2c-9d4e-2b861d13169c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:36.701898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11527a5a-e644-4b2c-9d4e-2b861d13169c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:36.702176 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11527a5a-e644-4b2c-9d4e-2b861d13169c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:36.702446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11527a5a-e644-4b2c-9d4e-2b861d13169c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:36.774691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11527a5a-e644-4b2c-9d4e-2b861d13169c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:36.776173 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11527a5a-e644-4b2c-9d4e-2b861d13169c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:36.837003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:36.879354 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 103/206] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:36.885293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd37bd-6ec9-4286-8263-6e74e580c1f4 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:36.995300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd37bd-6ec9-4286-8263-6e74e580c1f4 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:37.031053 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f98ef7-344f-4e9f-9ce7-62ce7dcd52cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:37.031672 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f98ef7-344f-4e9f-9ce7-62ce7dcd52cf None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:37.031938 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f98ef7-344f-4e9f-9ce7-62ce7dcd52cf None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:37.032210 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f98ef7-344f-4e9f-9ce7-62ce7dcd52cf None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:37.033345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f98ef7-344f-4e9f-9ce7-62ce7dcd52cf None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:37.034781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f98ef7-344f-4e9f-9ce7-62ce7dcd52cf None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:37.036494 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 104/207] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:36 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:37.043218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11527a5a-e644-4b2c-9d4e-2b861d13169c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:37.236677 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11527a5a-e644-4b2c-9d4e-2b861d13169c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:37.271385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30cdc4f-6cb7-4311-a813-612eb75a7a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:37.271911 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cdc4f-6cb7-4311-a813-612eb75a7a79 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:37.272188 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cdc4f-6cb7-4311-a813-612eb75a7a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:37.272458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cdc4f-6cb7-4311-a813-612eb75a7a79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:37.273261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30cdc4f-6cb7-4311-a813-612eb75a7a79 None admin] RBAC: Adding query filter params (role.id=bf2de8c80c5e4be79e1ea7adc3cbfc56, scope.project.id=4fe97b975d834d83a6733a0b2591bba4, user.id=aaf97666c16d4c7684bda4f48b7cd610) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:37.273623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30cdc4f-6cb7-4311-a813-612eb75a7a79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:37.275077 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30cdc4f-6cb7-4311-a813-612eb75a7a79 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:37.281557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 104/208] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:37 2022] GET /identity/v3/role_assignments?user.id=aaf97666c16d4c7684bda4f48b7cd610&scope.project.id=4fe97b975d834d83a6733a0b2591bba4&role.id=bf2de8c80c5e4be79e1ea7adc3cbfc56 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:37.860888 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310bc1d-0abf-494f-b25c-3490b0c1b64d None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:37.861349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310bc1d-0abf-494f-b25c-3490b0c1b64d None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:37.861721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310bc1d-0abf-494f-b25c-3490b0c1b64d None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:37.863400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 105/209] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:37.874169 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ce39d-6a14-41ae-950d-01141a7fafb1 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:37.874578 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ce39d-6a14-41ae-950d-01141a7fafb1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:37.874954 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ce39d-6a14-41ae-950d-01141a7fafb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:37.896754 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-696ce39d-6a14-41ae-950d-01141a7fafb1 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:37.897935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-696ce39d-6a14-41ae-950d-01141a7fafb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:37.916271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:37.982562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 105/210] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:38.069128 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b8ee7-ce5a-4a52-a705-a5fa9a14af91 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:38.069629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b8ee7-ce5a-4a52-a705-a5fa9a14af91 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:38.069629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b8ee7-ce5a-4a52-a705-a5fa9a14af91 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:38.071754 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 106/211] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:38.076949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28c8e2-89ab-4a64-b282-4738d1d18c63 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:38.077469 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28c8e2-89ab-4a64-b282-4738d1d18c63 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:38.077949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28c8e2-89ab-4a64-b282-4738d1d18c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:38.096574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-1c28c8e2-89ab-4a64-b282-4738d1d18c63 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:38.098140 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-1c28c8e2-89ab-4a64-b282-4738d1d18c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:38.167614 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 106/212] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:38.252181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b8ee7-ce5a-4a52-a705-a5fa9a14af91 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:38.257130 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:38.287139 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf8ae5-5992-484e-93cd-52cf4de7ee4a None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:38.287642 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf8ae5-5992-484e-93cd-52cf4de7ee4a None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:38.287742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf8ae5-5992-484e-93cd-52cf4de7ee4a None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:38.289552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 107/213] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:38.295405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c50027-185a-431e-a0b0-9a4ac09f2f30 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:38.295405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c50027-185a-431e-a0b0-9a4ac09f2f30 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:38.295617 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c50027-185a-431e-a0b0-9a4ac09f2f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:38.314165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-68c50027-185a-431e-a0b0-9a4ac09f2f30 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:38.315362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-68c50027-185a-431e-a0b0-9a4ac09f2f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:38.380578 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b8ee7-ce5a-4a52-a705-a5fa9a14af91 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:38.384202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 108/214] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:00:38.416599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6685e6-be17-4819-8d93-b4e3a06b8286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:38.417580 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6685e6-be17-4819-8d93-b4e3a06b8286 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:38.418062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6685e6-be17-4819-8d93-b4e3a06b8286 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:38.418522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6685e6-be17-4819-8d93-b4e3a06b8286 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:38.419942 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6685e6-be17-4819-8d93-b4e3a06b8286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:38.421616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6685e6-be17-4819-8d93-b4e3a06b8286 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:38.423494 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 107/215] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:38.432511 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c50027-185a-431e-a0b0-9a4ac09f2f30 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:38.442015 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:38.444270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6685e6-be17-4819-8d93-b4e3a06b8286 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:38.536907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:38.542661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c50027-185a-431e-a0b0-9a4ac09f2f30 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:38.579276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6685e6-be17-4819-8d93-b4e3a06b8286 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:38.584781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b60a6a-e166-4034-817f-2909638d4a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:38.585508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b60a6a-e166-4034-817f-2909638d4a99 None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:38.585954 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b60a6a-e166-4034-817f-2909638d4a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:38.586356 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b60a6a-e166-4034-817f-2909638d4a99 None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:38.587094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b60a6a-e166-4034-817f-2909638d4a99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:38.588790 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b60a6a-e166-4034-817f-2909638d4a99 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:38.618054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3bb90-898f-4ca8-9edc-c52c7c412f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:38.622119 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3bb90-898f-4ca8-9edc-c52c7c412f46 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:38.622119 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3bb90-898f-4ca8-9edc-c52c7c412f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:38.622119 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3bb90-898f-4ca8-9edc-c52c7c412f46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:38.629839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3bb90-898f-4ca8-9edc-c52c7c412f46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-58d3bb90-898f-4ca8-9edc-c52c7c412f46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:38.635823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:38.636738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:38.637567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:38.637567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:38.637567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:38.637567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:38.637567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:38.637567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:38.637567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:38.637567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:38.637567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:38.637567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:38.637567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:38.637567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:38.637567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 108/216] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:00:38.644016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3bb90-898f-4ca8-9edc-c52c7c412f46 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:38.695411 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 109/217] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:00:38 2022] POST /identity/v3/users => generated 297 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:00:38.709748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b60a6a-e166-4034-817f-2909638d4a99 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:38.749242 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3bb90-898f-4ca8-9edc-c52c7c412f46 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:38.781677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:38.786732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:38.788389 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d13d8-9666-48b5-8562-1b596ced4e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:38.789662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d13d8-9666-48b5-8562-1b596ced4e0c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:38.790111 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d13d8-9666-48b5-8562-1b596ced4e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:38.796186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d13d8-9666-48b5-8562-1b596ced4e0c None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:38.796186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d13d8-9666-48b5-8562-1b596ced4e0c None admin] RBAC: Adding query filter params (name=member) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:38.796186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d13d8-9666-48b5-8562-1b596ced4e0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:38.796186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d13d8-9666-48b5-8562-1b596ced4e0c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:38.815218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 109/218] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:38.816969 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644ae20-74b4-4cbd-967f-9948c479aa56 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:38.817388 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644ae20-74b4-4cbd-967f-9948c479aa56 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:38.817789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644ae20-74b4-4cbd-967f-9948c479aa56 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:38.819408 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 110/219] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:38.820447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644ae20-74b4-4cbd-967f-9948c479aa56 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:38.836220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b60a6a-e166-4034-817f-2909638d4a99 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:38.872522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dbd50-3b24-4250-a6fe-27ec6680c91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:38.873701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dbd50-3b24-4250-a6fe-27ec6680c91e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:38.873981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dbd50-3b24-4250-a6fe-27ec6680c91e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:38.874913 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dbd50-3b24-4250-a6fe-27ec6680c91e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:38.886663 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84dbd50-3b24-4250-a6fe-27ec6680c91e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-e84dbd50-3b24-4250-a6fe-27ec6680c91e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:38.892683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:38.893571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:38.895096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:38.895096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:38.895096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:38.895096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:38.895096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:38.895096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:38.895096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:38.895096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:38.895096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:38.895096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:38.895096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:38.895096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:38.898836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 110/220] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:00:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:38.900187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f604b-35fb-4cdf-9676-989ce89046ad None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:38.900454 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f604b-35fb-4cdf-9676-989ce89046ad None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:38.900771 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f604b-35fb-4cdf-9676-989ce89046ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:38.920172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-649f604b-35fb-4cdf-9676-989ce89046ad None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:38.921414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-649f604b-35fb-4cdf-9676-989ce89046ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:38.959598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644ae20-74b4-4cbd-967f-9948c479aa56 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:39.004408 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84de48-abc0-49ea-b076-1cba95fca718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:39.005395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84de48-abc0-49ea-b076-1cba95fca718 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:39.005872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84de48-abc0-49ea-b076-1cba95fca718 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:39.006359 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84de48-abc0-49ea-b076-1cba95fca718 None admin] PATH_INFO: `/v3/users/c2d7a3cf077348be9e4b9d1e0690439e` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:39.008376 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 111/221] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:39.009237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f604b-35fb-4cdf-9676-989ce89046ad None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:39.022551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84de48-abc0-49ea-b076-1cba95fca718 None admin] RBAC: Authorizing `identity:get_user(user_id=c2d7a3cf077348be9e4b9d1e0690439e)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:39.025032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84de48-abc0-49ea-b076-1cba95fca718 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:39.027060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 111/222] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:38 2022] GET /identity/v3/users/c2d7a3cf077348be9e4b9d1e0690439e => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:39.033478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84de48-abc0-49ea-b076-1cba95fca718 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:39.101703 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:39.103582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:39.152791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84de48-abc0-49ea-b076-1cba95fca718 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:39.189479 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d1047-f91f-43c7-9585-18c5e3238a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:39.190097 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d1047-f91f-43c7-9585-18c5e3238a44 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:39.190380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d1047-f91f-43c7-9585-18c5e3238a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:39.190655 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d1047-f91f-43c7-9585-18c5e3238a44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:39.207946 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f604b-35fb-4cdf-9676-989ce89046ad None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:39.262224 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d1047-f91f-43c7-9585-18c5e3238a44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:39.263712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d1047-f91f-43c7-9585-18c5e3238a44 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:39.293905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0c916-2dce-4225-bd19-5e9252efc363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:39.295279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0c916-2dce-4225-bd19-5e9252efc363 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:39.295761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0c916-2dce-4225-bd19-5e9252efc363 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:39.296235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0c916-2dce-4225-bd19-5e9252efc363 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:39.297154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0c916-2dce-4225-bd19-5e9252efc363 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:39.297672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0c916-2dce-4225-bd19-5e9252efc363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:39.299255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0c916-2dce-4225-bd19-5e9252efc363 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:39.318477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:39.320302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 112/223] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:39.321169 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0c916-2dce-4225-bd19-5e9252efc363 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:39.377182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 112/224] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:39.378993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d1047-f91f-43c7-9585-18c5e3238a44 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:39.450392 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0c916-2dce-4225-bd19-5e9252efc363 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:39.471990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:39.495343 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab4162-a7d6-4861-8951-ffa966ba202d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:39.496355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab4162-a7d6-4861-8951-ffa966ba202d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:39.496887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab4162-a7d6-4861-8951-ffa966ba202d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:39.497377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab4162-a7d6-4861-8951-ffa966ba202d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:39.502860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d1047-f91f-43c7-9585-18c5e3238a44 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:39.510279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab4162-a7d6-4861-8951-ffa966ba202d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-ccab4162-a7d6-4861-8951-ffa966ba202d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:39.516094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:39.516980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:39.517741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:39.517741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:39.517741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:39.517741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:39.517741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:39.517741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:39.517741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:39.517741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:39.517741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:39.517741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:39.517741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:39.517741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:39.520064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 113/225] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:00:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:39.525740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab4162-a7d6-4861-8951-ffa966ba202d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:39.541849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ef7a7-0621-404b-8f74-ef6b60cdb3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:39.542570 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ef7a7-0621-404b-8f74-ef6b60cdb3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:39.542970 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ef7a7-0621-404b-8f74-ef6b60cdb3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:39.543257 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ef7a7-0621-404b-8f74-ef6b60cdb3e6 None admin] PATH_INFO: `/v3/users/c9421b5ced544e1591c7ab5d6204d5f0` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:39.544479 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ef7a7-0621-404b-8f74-ef6b60cdb3e6 None admin] RBAC: Authorizing `identity:get_user(user_id=c9421b5ced544e1591c7ab5d6204d5f0)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:39.546220 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ef7a7-0621-404b-8f74-ef6b60cdb3e6 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:39.547706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 113/226] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:39 2022] GET /identity/v3/users/c9421b5ced544e1591c7ab5d6204d5f0 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:39.548582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ef7a7-0621-404b-8f74-ef6b60cdb3e6 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:39.571338 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:39.637855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab4162-a7d6-4861-8951-ffa966ba202d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:39.653084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ef7a7-0621-404b-8f74-ef6b60cdb3e6 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:39.674214 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea59c0-bf4f-4fbd-b0ff-44f8e142d4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:39.675197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea59c0-bf4f-4fbd-b0ff-44f8e142d4ee None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:39.675675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea59c0-bf4f-4fbd-b0ff-44f8e142d4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:39.676154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea59c0-bf4f-4fbd-b0ff-44f8e142d4ee None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:39.677510 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea59c0-bf4f-4fbd-b0ff-44f8e142d4ee None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:39.679311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea59c0-bf4f-4fbd-b0ff-44f8e142d4ee None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:39.681687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 114/227] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:39 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:39.682761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea59c0-bf4f-4fbd-b0ff-44f8e142d4ee None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:39.690440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8ba84-1f01-48cf-a33a-d73340cef6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:39.691093 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8ba84-1f01-48cf-a33a-d73340cef6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:39.691379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8ba84-1f01-48cf-a33a-d73340cef6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:39.691650 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8ba84-1f01-48cf-a33a-d73340cef6c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:39.692514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8ba84-1f01-48cf-a33a-d73340cef6c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:39.693091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8ba84-1f01-48cf-a33a-d73340cef6c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:39.695992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8ba84-1f01-48cf-a33a-d73340cef6c3 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:39.705046 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 114/228] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:00:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:39.708678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8ba84-1f01-48cf-a33a-d73340cef6c3 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:39.739728 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:39.794880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:39.798408 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea59c0-bf4f-4fbd-b0ff-44f8e142d4ee None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:39.824551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8ba84-1f01-48cf-a33a-d73340cef6c3 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:39.837711 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb31e8-1e76-4c79-88d6-5e368c9808cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:39.838700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb31e8-1e76-4c79-88d6-5e368c9808cb None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:39.839176 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb31e8-1e76-4c79-88d6-5e368c9808cb None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:39.839690 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb31e8-1e76-4c79-88d6-5e368c9808cb None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:39.841144 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb31e8-1e76-4c79-88d6-5e368c9808cb None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:39.842905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb31e8-1e76-4c79-88d6-5e368c9808cb None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:39.844863 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 115/229] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:39 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:39.845810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb31e8-1e76-4c79-88d6-5e368c9808cb None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:39.861852 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8cfcf4-11ce-440d-910f-f6905cb06e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:39.862463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cfcf4-11ce-440d-910f-f6905cb06e85 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:39.862795 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cfcf4-11ce-440d-910f-f6905cb06e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:39.863074 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cfcf4-11ce-440d-910f-f6905cb06e85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:39.863858 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8cfcf4-11ce-440d-910f-f6905cb06e85 None admin] RBAC: Adding query filter params (role.id=8b2ee9b11a934b67aa3fd68c497fbdf7, scope.project.id=ad5d2677b95b428a93e499f4ced1b86b, user.id=c2d7a3cf077348be9e4b9d1e0690439e) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:39.864110 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8cfcf4-11ce-440d-910f-f6905cb06e85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:39.865591 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8cfcf4-11ce-440d-910f-f6905cb06e85 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:39.871824 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 115/230] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:39 2022] GET /identity/v3/role_assignments?user.id=c2d7a3cf077348be9e4b9d1e0690439e&scope.project.id=ad5d2677b95b428a93e499f4ced1b86b&role.id=8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:39.872838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8cfcf4-11ce-440d-910f-f6905cb06e85 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:39.956084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb31e8-1e76-4c79-88d6-5e368c9808cb None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:39.992703 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:40.003532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfd719-a5a9-4e32-9d10-eb2385361aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:40.008004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfd719-a5a9-4e32-9d10-eb2385361aba None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:40.008004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfd719-a5a9-4e32-9d10-eb2385361aba None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:40.008004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfd719-a5a9-4e32-9d10-eb2385361aba None admin] PATH_INFO: `/v3/users/c9421b5ced544e1591c7ab5d6204d5f0` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:40.008858 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfd719-a5a9-4e32-9d10-eb2385361aba None admin] RBAC: Authorizing `identity:get_user(user_id=c9421b5ced544e1591c7ab5d6204d5f0)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:40.010512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfd719-a5a9-4e32-9d10-eb2385361aba None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:40.012438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 116/231] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:39 2022] GET /identity/v3/users/c9421b5ced544e1591c7ab5d6204d5f0 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:40.022099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfd719-a5a9-4e32-9d10-eb2385361aba None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:40.036479 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8cfcf4-11ce-440d-910f-f6905cb06e85 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:40.091540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc30f9f-48b7-486f-84a9-46a819dcb269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:40.092226 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc30f9f-48b7-486f-84a9-46a819dcb269 None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:40.092652 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc30f9f-48b7-486f-84a9-46a819dcb269 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:40.092954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc30f9f-48b7-486f-84a9-46a819dcb269 None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b/users/c9421b5ced544e1591c7ab5d6204d5f0/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:40.095048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc30f9f-48b7-486f-84a9-46a819dcb269 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad5d2677b95b428a93e499f4ced1b86b, user_id=c9421b5ced544e1591c7ab5d6204d5f0, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:40.096758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc30f9f-48b7-486f-84a9-46a819dcb269 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:40.178724 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfd719-a5a9-4e32-9d10-eb2385361aba None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:40.187556 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 116/232] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:00:39 2022] PUT /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b/users/c9421b5ced544e1591c7ab5d6204d5f0/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:00:40.229949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20541f-d9d6-470d-8fa2-90a8b65dfcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:40.230515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20541f-d9d6-470d-8fa2-90a8b65dfcf9 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:40.230800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20541f-d9d6-470d-8fa2-90a8b65dfcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:40.231087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20541f-d9d6-470d-8fa2-90a8b65dfcf9 None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:40.232220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20541f-d9d6-470d-8fa2-90a8b65dfcf9 None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:40.233763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20541f-d9d6-470d-8fa2-90a8b65dfcf9 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:40.235829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 117/233] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:40 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:40.241416 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc30f9f-48b7-486f-84a9-46a819dcb269 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:40.257074 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:40.395734 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc30f9f-48b7-486f-84a9-46a819dcb269 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:40.432665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8065e-a1ee-40b5-83c8-8c04abcdab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:40.433324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8065e-a1ee-40b5-83c8-8c04abcdab5b None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:40.433604 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8065e-a1ee-40b5-83c8-8c04abcdab5b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:40.433877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8065e-a1ee-40b5-83c8-8c04abcdab5b None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b/users/c9421b5ced544e1591c7ab5d6204d5f0/roles/b5e411fc96434df69c6cc1df951b3e6f` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:40.436612 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8065e-a1ee-40b5-83c8-8c04abcdab5b None admin] RBAC: Authorizing `identity:create_grant(project_id=ad5d2677b95b428a93e499f4ced1b86b, user_id=c9421b5ced544e1591c7ab5d6204d5f0, role_id=b5e411fc96434df69c6cc1df951b3e6f)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:40.438096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8065e-a1ee-40b5-83c8-8c04abcdab5b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:40.536529 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 117/234] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:00:40 2022] PUT /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b/users/c9421b5ced544e1591c7ab5d6204d5f0/roles/b5e411fc96434df69c6cc1df951b3e6f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:00:40.988035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b87d43-6ca0-4dc7-b5d1-649db9f68562 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:40.988563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b87d43-6ca0-4dc7-b5d1-649db9f68562 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:40.988563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b87d43-6ca0-4dc7-b5d1-649db9f68562 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:40.990165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 118/235] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:40.995511 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003dfe1-3205-4a3f-8dc5-9007d34843fe None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:40.995802 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003dfe1-3205-4a3f-8dc5-9007d34843fe None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:40.996083 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003dfe1-3205-4a3f-8dc5-9007d34843fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:41.021631 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-7003dfe1-3205-4a3f-8dc5-9007d34843fe None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:41.022818 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-7003dfe1-3205-4a3f-8dc5-9007d34843fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:41.033853 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732130a-b12c-4cec-9844-432f15ca6e30 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:41.033853 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732130a-b12c-4cec-9844-432f15ca6e30 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:41.033853 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732130a-b12c-4cec-9844-432f15ca6e30 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:41.033853 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 119/236] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:00:41.046372 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc8fb3-8eb5-4b36-adf2-879ab2f4caca None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:41.046935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc8fb3-8eb5-4b36-adf2-879ab2f4caca None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:41.047397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc8fb3-8eb5-4b36-adf2-879ab2f4caca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:41.066793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-71fc8fb3-8eb5-4b36-adf2-879ab2f4caca None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:41.068372 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-71fc8fb3-8eb5-4b36-adf2-879ab2f4caca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:41.129015 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:41.146667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 118/237] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:41.149642 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:41.170206 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 120/238] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:41.597136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003dfe1-3205-4a3f-8dc5-9007d34843fe None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:41.626731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc8fb3-8eb5-4b36-adf2-879ab2f4caca None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:41.731746 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc8fb3-8eb5-4b36-adf2-879ab2f4caca None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:41.748228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003dfe1-3205-4a3f-8dc5-9007d34843fe None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:41.768229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84c63b-caa0-4858-80ef-9048f06181c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:41.768898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84c63b-caa0-4858-80ef-9048f06181c5 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:41.769182 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84c63b-caa0-4858-80ef-9048f06181c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:41.769455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84c63b-caa0-4858-80ef-9048f06181c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:41.770560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84c63b-caa0-4858-80ef-9048f06181c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:41.772012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84c63b-caa0-4858-80ef-9048f06181c5 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:41.774038 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 121/239] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:41.784373 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84c63b-caa0-4858-80ef-9048f06181c5 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:41.828842 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c2335-033e-4647-8055-a44b393fcd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:41.829804 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c2335-033e-4647-8055-a44b393fcd9a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:41.830316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c2335-033e-4647-8055-a44b393fcd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:41.830790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c2335-033e-4647-8055-a44b393fcd9a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:41.837451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:41.865575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c2335-033e-4647-8055-a44b393fcd9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-fb8c2335-033e-4647-8055-a44b393fcd9a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:41.870214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:41.871092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:41.871861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:41.871861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:41.871861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:41.871861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:41.871861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:41.871861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:41.871861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:41.871861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:41.871861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:41.871861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:41.871861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:41.871861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:41.871861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 119/240] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:41.881626 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c2335-033e-4647-8055-a44b393fcd9a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:41.895604 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84c63b-caa0-4858-80ef-9048f06181c5 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:41.934244 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a73115-c01c-4f1a-9c31-554a91aa8b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:41.935231 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a73115-c01c-4f1a-9c31-554a91aa8b47 None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:41.935738 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a73115-c01c-4f1a-9c31-554a91aa8b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:41.936207 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a73115-c01c-4f1a-9c31-554a91aa8b47 None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:41.937098 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a73115-c01c-4f1a-9c31-554a91aa8b47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:41.938766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a73115-c01c-4f1a-9c31-554a91aa8b47 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:41.984873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:42.032469 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c2335-033e-4647-8055-a44b393fcd9a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:42.048265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 122/241] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:00:41 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:00:42.082063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24afcb-02cb-4b50-bd2a-8750daf3bf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:42.082986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24afcb-02cb-4b50-bd2a-8750daf3bf1a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:42.083439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24afcb-02cb-4b50-bd2a-8750daf3bf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:42.083910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24afcb-02cb-4b50-bd2a-8750daf3bf1a None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:42.084800 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24afcb-02cb-4b50-bd2a-8750daf3bf1a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:42.085328 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24afcb-02cb-4b50-bd2a-8750daf3bf1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:42.087486 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24afcb-02cb-4b50-bd2a-8750daf3bf1a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:42.097736 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 120/242] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:42.105474 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a73115-c01c-4f1a-9c31-554a91aa8b47 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:42.195632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdfebc-337c-40d8-958d-400a7ff52aaf None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:42.196912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdfebc-337c-40d8-958d-400a7ff52aaf None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:42.197364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdfebc-337c-40d8-958d-400a7ff52aaf None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:42.199890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 121/243] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:00:42.206050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c9d32-c830-4c91-93ba-a9459794d6d0 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:42.206629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c9d32-c830-4c91-93ba-a9459794d6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:42.207078 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c9d32-c830-4c91-93ba-a9459794d6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:42.227511 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a73115-c01c-4f1a-9c31-554a91aa8b47 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:42.230326 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:42.244878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-eb2c9d32-c830-4c91-93ba-a9459794d6d0 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:42.246404 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-eb2c9d32-c830-4c91-93ba-a9459794d6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:42.272822 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7646775-86db-427b-b767-5678137189b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:42.274058 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7646775-86db-427b-b767-5678137189b5 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:42.274335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7646775-86db-427b-b767-5678137189b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:42.274812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7646775-86db-427b-b767-5678137189b5 None admin] PATH_INFO: `/v3/users/d9a9aab4d92f4d96a7046582cdc3661f` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:42.290231 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7646775-86db-427b-b767-5678137189b5 None admin] RBAC: Authorizing `identity:get_user(user_id=d9a9aab4d92f4d96a7046582cdc3661f)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:42.291708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7646775-86db-427b-b767-5678137189b5 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:42.293645 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 123/244] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:42 2022] GET /identity/v3/users/d9a9aab4d92f4d96a7046582cdc3661f => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:42.299247 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7646775-86db-427b-b767-5678137189b5 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:42.323071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 122/245] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:00:42.407650 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7646775-86db-427b-b767-5678137189b5 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:42.443732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a22ad5-fead-480a-a69e-f279d2f996d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:42.444335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a22ad5-fead-480a-a69e-f279d2f996d7 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:42.444610 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a22ad5-fead-480a-a69e-f279d2f996d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:42.445060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a22ad5-fead-480a-a69e-f279d2f996d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:42.478156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:42.516684 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a22ad5-fead-480a-a69e-f279d2f996d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:42.518271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a22ad5-fead-480a-a69e-f279d2f996d7 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:42.619600 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a131be9-617a-4586-9ea9-1d71871ac7f1 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:42.620106 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a131be9-617a-4586-9ea9-1d71871ac7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:42.620209 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a131be9-617a-4586-9ea9-1d71871ac7f1 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:42.621778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 123/246] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:42.622514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 124/247] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 12 12:00:42.626869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98875798-3676-4bbc-be21-d30012508e66 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:42.627218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98875798-3676-4bbc-be21-d30012508e66 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:42.627498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98875798-3676-4bbc-be21-d30012508e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:42.628495 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a22ad5-fead-480a-a69e-f279d2f996d7 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:42.649955 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-98875798-3676-4bbc-be21-d30012508e66 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:42.651152 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-98875798-3676-4bbc-be21-d30012508e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:42.699965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:42.709307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:42.743507 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 124/248] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:42.743507 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98875798-3676-4bbc-be21-d30012508e66 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:42.753078 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a22ad5-fead-480a-a69e-f279d2f996d7 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:42.794748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9d780-8c72-4485-8c42-cb13d2b492f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:42.795968 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9d780-8c72-4485-8c42-cb13d2b492f0 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:42.799576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9d780-8c72-4485-8c42-cb13d2b492f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:42.800057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9d780-8c72-4485-8c42-cb13d2b492f0 None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:42.802369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9d780-8c72-4485-8c42-cb13d2b492f0 None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:42.808702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9d780-8c72-4485-8c42-cb13d2b492f0 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:42.811209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 125/249] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:42 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:42.818223 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9d780-8c72-4485-8c42-cb13d2b492f0 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:42.868890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98875798-3676-4bbc-be21-d30012508e66 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:42.927679 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc320ff4-7e31-453b-9c49-595ceea11059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:42.928355 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc320ff4-7e31-453b-9c49-595ceea11059 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:42.928695 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc320ff4-7e31-453b-9c49-595ceea11059 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:42.929048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc320ff4-7e31-453b-9c49-595ceea11059 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:42.949887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9d780-8c72-4485-8c42-cb13d2b492f0 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:42.955658 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc320ff4-7e31-453b-9c49-595ceea11059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-bc320ff4-7e31-453b-9c49-595ceea11059 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:42.961036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:42.961951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:42.962745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:42.962745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:42.962745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:42.962745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:42.962745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:42.962745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:42.962745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:42.962745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:42.962745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:42.962745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:42.962745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:42.962745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:42.967129 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 125/250] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:42.968406 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc320ff4-7e31-453b-9c49-595ceea11059 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:42.987820 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:43.002493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202863c-20bc-4fbf-bf12-f65859a656d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:43.002493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202863c-20bc-4fbf-bf12-f65859a656d9 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:43.002493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202863c-20bc-4fbf-bf12-f65859a656d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:43.002493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202863c-20bc-4fbf-bf12-f65859a656d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:43.002855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202863c-20bc-4fbf-bf12-f65859a656d9 None admin] RBAC: Adding query filter params (role.id=bf2de8c80c5e4be79e1ea7adc3cbfc56, scope.project.id=ad5d2677b95b428a93e499f4ced1b86b, user.id=d9a9aab4d92f4d96a7046582cdc3661f) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:43.002855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202863c-20bc-4fbf-bf12-f65859a656d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:43.002855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202863c-20bc-4fbf-bf12-f65859a656d9 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:43.007605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 126/251] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:42 2022] GET /identity/v3/role_assignments?user.id=d9a9aab4d92f4d96a7046582cdc3661f&scope.project.id=ad5d2677b95b428a93e499f4ced1b86b&role.id=bf2de8c80c5e4be79e1ea7adc3cbfc56 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:43.050105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed630bdb-5006-4ba9-a03d-4e56f9d0c7e3 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:43.051129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed630bdb-5006-4ba9-a03d-4e56f9d0c7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:43.051623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed630bdb-5006-4ba9-a03d-4e56f9d0c7e3 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:43.053531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 127/252] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:43.059742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6245fc7-3250-4a15-9952-d4249cb5c60a None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:43.060335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6245fc7-3250-4a15-9952-d4249cb5c60a None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:43.060836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6245fc7-3250-4a15-9952-d4249cb5c60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:43.082700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:43.087435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-d6245fc7-3250-4a15-9952-d4249cb5c60a None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:43.088716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-d6245fc7-3250-4a15-9952-d4249cb5c60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:43.175761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc320ff4-7e31-453b-9c49-595ceea11059 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:43.213628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b6f8c-9a71-4f92-bbbe-6c43d60ac128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:43.214244 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b6f8c-9a71-4f92-bbbe-6c43d60ac128 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:43.214523 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b6f8c-9a71-4f92-bbbe-6c43d60ac128 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:43.214800 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b6f8c-9a71-4f92-bbbe-6c43d60ac128 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:43.214895 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 128/253] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:43.215480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b6f8c-9a71-4f92-bbbe-6c43d60ac128 None admin] RBAC: Adding query filter params (name=member) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:43.215773 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b6f8c-9a71-4f92-bbbe-6c43d60ac128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:43.215868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6245fc7-3250-4a15-9952-d4249cb5c60a None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:43.217293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b6f8c-9a71-4f92-bbbe-6c43d60ac128 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:43.227111 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 126/254] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:43.240762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b6f8c-9a71-4f92-bbbe-6c43d60ac128 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:43.430018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6245fc7-3250-4a15-9952-d4249cb5c60a None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:43.452557 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:43.458655 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b6f8c-9a71-4f92-bbbe-6c43d60ac128 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:43.481676 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:43.502302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97277d-8f37-4906-993d-a1a7574cd79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:43.502943 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97277d-8f37-4906-993d-a1a7574cd79a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:43.503225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97277d-8f37-4906-993d-a1a7574cd79a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:43.503499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97277d-8f37-4906-993d-a1a7574cd79a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:43.530633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28b1f0-95a0-4aab-8632-8b5615bf3b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:43.531501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28b1f0-95a0-4aab-8632-8b5615bf3b15 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:43.531919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28b1f0-95a0-4aab-8632-8b5615bf3b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:43.532326 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28b1f0-95a0-4aab-8632-8b5615bf3b15 None admin] PATH_INFO: `/v3/users/c2d7a3cf077348be9e4b9d1e0690439e` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:43.537335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28b1f0-95a0-4aab-8632-8b5615bf3b15 None admin] RBAC: Authorizing `identity:get_user(user_id=c2d7a3cf077348be9e4b9d1e0690439e)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:43.537567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97277d-8f37-4906-993d-a1a7574cd79a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:43.539582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28b1f0-95a0-4aab-8632-8b5615bf3b15 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:43.541428 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 127/255] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:43 2022] GET /identity/v3/users/c2d7a3cf077348be9e4b9d1e0690439e => generated 297 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-0f97277d-8f37-4906-993d-a1a7574cd79a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:43.542410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:43.543429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:43.544357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:43.544357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:43.544357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:43.544357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:43.544357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:43.544357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:43.544357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:43.544357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:43.544357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:43.544357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:43.544357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:43.544357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:43.544357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 129/256] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:00:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:43.547131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28b1f0-95a0-4aab-8632-8b5615bf3b15 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:43.550351 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97277d-8f37-4906-993d-a1a7574cd79a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:43.740345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28b1f0-95a0-4aab-8632-8b5615bf3b15 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:43.753287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97277d-8f37-4906-993d-a1a7574cd79a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:43.807809 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1673742-4cbc-4908-b044-f2de327ab63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:43.808447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1673742-4cbc-4908-b044-f2de327ab63a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:43.808746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1673742-4cbc-4908-b044-f2de327ab63a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:43.809020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1673742-4cbc-4908-b044-f2de327ab63a None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:43.810770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1673742-4cbc-4908-b044-f2de327ab63a None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:43.813660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1673742-4cbc-4908-b044-f2de327ab63a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:43.813660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 128/257] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:43 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:43.814237 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1673742-4cbc-4908-b044-f2de327ab63a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:43.817595 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3acb6-f82e-47d1-b6dc-1d429052cd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:43.818438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3acb6-f82e-47d1-b6dc-1d429052cd6f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:43.818541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3acb6-f82e-47d1-b6dc-1d429052cd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:43.818734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3acb6-f82e-47d1-b6dc-1d429052cd6f None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:43.819409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3acb6-f82e-47d1-b6dc-1d429052cd6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:43.819780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3acb6-f82e-47d1-b6dc-1d429052cd6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:43.821176 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3acb6-f82e-47d1-b6dc-1d429052cd6f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:43.839610 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 130/258] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:43.840254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3acb6-f82e-47d1-b6dc-1d429052cd6f None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:43.892805 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:43.925974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:44.001721 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3acb6-f82e-47d1-b6dc-1d429052cd6f None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:44.038097 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24654e5-d116-4e42-bfbd-29796a79faaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:44.039449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24654e5-d116-4e42-bfbd-29796a79faaa None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:44.039557 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1673742-4cbc-4908-b044-f2de327ab63a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:44.039734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24654e5-d116-4e42-bfbd-29796a79faaa None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:44.040003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24654e5-d116-4e42-bfbd-29796a79faaa None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b/users/c2d7a3cf077348be9e4b9d1e0690439e/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:44.042251 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24654e5-d116-4e42-bfbd-29796a79faaa None admin] RBAC: Authorizing `identity:create_grant(project_id=ad5d2677b95b428a93e499f4ced1b86b, user_id=c2d7a3cf077348be9e4b9d1e0690439e, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:44.043710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24654e5-d116-4e42-bfbd-29796a79faaa None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:44.084003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62524bc-9903-4731-90a9-c9602b67d576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:44.088255 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62524bc-9903-4731-90a9-c9602b67d576 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:44.088255 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62524bc-9903-4731-90a9-c9602b67d576 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:44.088255 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62524bc-9903-4731-90a9-c9602b67d576 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:44.099179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62524bc-9903-4731-90a9-c9602b67d576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-c62524bc-9903-4731-90a9-c9602b67d576 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:44.104662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:44.106876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:44.108793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:44.108793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:44.108793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:44.108793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:44.108793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:44.108793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:44.108793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:44.108793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:44.108793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:44.108793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:44.108793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:44.108793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:44.108793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 129/259] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:00:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:44.108793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62524bc-9903-4731-90a9-c9602b67d576 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:44.142958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 131/260] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:00:43 2022] PUT /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b/users/c2d7a3cf077348be9e4b9d1e0690439e/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:00:44.143633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24654e5-d116-4e42-bfbd-29796a79faaa None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:44.211257 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:44.219062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62524bc-9903-4731-90a9-c9602b67d576 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:44.263426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65eb77f-d607-4826-b288-4b455e6dd8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:44.264291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65eb77f-d607-4826-b288-4b455e6dd8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:44.264844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65eb77f-d607-4826-b288-4b455e6dd8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:44.265263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65eb77f-d607-4826-b288-4b455e6dd8b7 None admin] PATH_INFO: `/v3/users/c9421b5ced544e1591c7ab5d6204d5f0` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:44.266602 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65eb77f-d607-4826-b288-4b455e6dd8b7 None admin] RBAC: Authorizing `identity:get_user(user_id=c9421b5ced544e1591c7ab5d6204d5f0)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:44.268271 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65eb77f-d607-4826-b288-4b455e6dd8b7 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:44.270152 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 130/261] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:44 2022] GET /identity/v3/users/c9421b5ced544e1591c7ab5d6204d5f0 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:44.276341 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65eb77f-d607-4826-b288-4b455e6dd8b7 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:44.387127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24654e5-d116-4e42-bfbd-29796a79faaa None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:44.390240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65eb77f-d607-4826-b288-4b455e6dd8b7 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:44.399596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:44.431128 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8174c-6c7e-41b9-b501-f305d8cd7af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:44.432241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8174c-6c7e-41b9-b501-f305d8cd7af2 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:44.432686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8174c-6c7e-41b9-b501-f305d8cd7af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:44.433102 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8174c-6c7e-41b9-b501-f305d8cd7af2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:44.468713 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:44.485413 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eabb8-40da-4985-a106-705c32a96d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:44.486047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eabb8-40da-4985-a106-705c32a96d80 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:44.486332 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eabb8-40da-4985-a106-705c32a96d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:44.486616 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eabb8-40da-4985-a106-705c32a96d80 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:44.487297 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15eabb8-40da-4985-a106-705c32a96d80 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:44.487600 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15eabb8-40da-4985-a106-705c32a96d80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:44.489181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15eabb8-40da-4985-a106-705c32a96d80 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:44.504398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 132/262] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:00:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:44.510449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eabb8-40da-4985-a106-705c32a96d80 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:44.513808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8174c-6c7e-41b9-b501-f305d8cd7af2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:44.515533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8174c-6c7e-41b9-b501-f305d8cd7af2 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:44.618767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 131/263] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 12 12:00:44.624527 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8174c-6c7e-41b9-b501-f305d8cd7af2 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:44.689034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:44.729259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8174c-6c7e-41b9-b501-f305d8cd7af2 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:44.737859 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eabb8-40da-4985-a106-705c32a96d80 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:44.765204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aaf0d4-2ce8-4ea3-8176-865a503e1d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:44.766377 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aaf0d4-2ce8-4ea3-8176-865a503e1d27 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:44.766800 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aaf0d4-2ce8-4ea3-8176-865a503e1d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:44.767212 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aaf0d4-2ce8-4ea3-8176-865a503e1d27 None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:44.768497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aaf0d4-2ce8-4ea3-8176-865a503e1d27 None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:44.770123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aaf0d4-2ce8-4ea3-8176-865a503e1d27 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:44.771928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 132/264] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:44 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:44.778413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aaf0d4-2ce8-4ea3-8176-865a503e1d27 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:44.808098 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4d775-8608-481d-8257-5068a947280d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:44.808775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4d775-8608-481d-8257-5068a947280d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:44.809121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4d775-8608-481d-8257-5068a947280d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:44.809318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4d775-8608-481d-8257-5068a947280d None admin] PATH_INFO: `/v3/users/c9421b5ced544e1591c7ab5d6204d5f0` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:44.810687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4d775-8608-481d-8257-5068a947280d None admin] RBAC: Authorizing `identity:get_user(user_id=c9421b5ced544e1591c7ab5d6204d5f0)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:44.812015 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4d775-8608-481d-8257-5068a947280d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:44.813633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 133/265] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:44 2022] GET /identity/v3/users/c9421b5ced544e1591c7ab5d6204d5f0 => generated 283 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:44.819302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4d775-8608-481d-8257-5068a947280d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:44.881049 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:44.921754 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aaf0d4-2ce8-4ea3-8176-865a503e1d27 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:44.982940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:44.988393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bfe22-6f8e-46c3-95f2-87d999a07853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:44.989141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bfe22-6f8e-46c3-95f2-87d999a07853 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:44.992761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bfe22-6f8e-46c3-95f2-87d999a07853 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:44.992761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bfe22-6f8e-46c3-95f2-87d999a07853 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:44.992761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bfe22-6f8e-46c3-95f2-87d999a07853 None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=ad5d2677b95b428a93e499f4ced1b86b, user.id=c9421b5ced544e1591c7ab5d6204d5f0) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:44.992761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bfe22-6f8e-46c3-95f2-87d999a07853 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:44.992761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bfe22-6f8e-46c3-95f2-87d999a07853 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:45.006966 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 133/266] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:44 2022] GET /identity/v3/role_assignments?user.id=c9421b5ced544e1591c7ab5d6204d5f0&scope.project.id=ad5d2677b95b428a93e499f4ced1b86b&role.id=906e3a8a6c7540a38f629f2567133347 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:45.010613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4d775-8608-481d-8257-5068a947280d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:45.072503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8770b-9e68-4906-9774-86f465f5ddc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:45.073070 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8770b-9e68-4906-9774-86f465f5ddc6 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:45.073357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8770b-9e68-4906-9774-86f465f5ddc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:45.073632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8770b-9e68-4906-9774-86f465f5ddc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:45.164094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8770b-9e68-4906-9774-86f465f5ddc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:45.169896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8770b-9e68-4906-9774-86f465f5ddc6 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:45.181670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b33d-7561-4506-a7e5-34d42f58300c None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:45.181963 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b33d-7561-4506-a7e5-34d42f58300c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:45.182224 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b33d-7561-4506-a7e5-34d42f58300c None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:45.183705 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 134/267] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:00:45.188853 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ae487-7a5d-4be6-87af-304d1d67ad68 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:45.189138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ae487-7a5d-4be6-87af-304d1d67ad68 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:45.189400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ae487-7a5d-4be6-87af-304d1d67ad68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:45.216333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:45.229834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-295ae487-7a5d-4be6-87af-304d1d67ad68 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:45.238879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-295ae487-7a5d-4be6-87af-304d1d67ad68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:45.341169 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 134/268] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:45.347730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8770b-9e68-4906-9774-86f465f5ddc6 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:45.359572 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 135/269] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:00:45.440322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63276791-39e7-4900-9196-53eda92ce0e9 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:45.440620 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63276791-39e7-4900-9196-53eda92ce0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:45.440890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63276791-39e7-4900-9196-53eda92ce0e9 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:45.442368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 136/270] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:00:45.452991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ec181-08d0-4db4-9d38-0a35de462b3f None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:45.453274 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ec181-08d0-4db4-9d38-0a35de462b3f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:45.453546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ec181-08d0-4db4-9d38-0a35de462b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:45.496484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8770b-9e68-4906-9774-86f465f5ddc6 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:45.505644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-3a8ec181-08d0-4db4-9d38-0a35de462b3f None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:45.506851 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-3a8ec181-08d0-4db4-9d38-0a35de462b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:45.540120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2521e1-804e-4276-8f8d-d3bb89493a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:45.541155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2521e1-804e-4276-8f8d-d3bb89493a18 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:45.541799 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2521e1-804e-4276-8f8d-d3bb89493a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:45.542299 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2521e1-804e-4276-8f8d-d3bb89493a18 None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:45.543688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2521e1-804e-4276-8f8d-d3bb89493a18 None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:45.545611 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2521e1-804e-4276-8f8d-d3bb89493a18 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:45.547666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 135/271] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:45 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:45.554124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2521e1-804e-4276-8f8d-d3bb89493a18 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:45.569001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:45.634732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 137/272] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:00:45.636544 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f9dda-8f8d-4227-8ee3-353886c991fd None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:45.637048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f9dda-8f8d-4227-8ee3-353886c991fd None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:45.638035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f9dda-8f8d-4227-8ee3-353886c991fd None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:45.640003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 138/273] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:45.646164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43924e-701f-4c48-a152-3ed668ec035d None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:45.646636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43924e-701f-4c48-a152-3ed668ec035d None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:45.647079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43924e-701f-4c48-a152-3ed668ec035d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:45.667894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-ec43924e-701f-4c48-a152-3ed668ec035d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:45.669798 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-ec43924e-701f-4c48-a152-3ed668ec035d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:45.695095 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2521e1-804e-4276-8f8d-d3bb89493a18 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:45.732281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:45.746386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672cec6-bc83-4fd5-ba21-caf7637709a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:45.747016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672cec6-bc83-4fd5-ba21-caf7637709a9 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:45.747297 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672cec6-bc83-4fd5-ba21-caf7637709a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:45.747571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672cec6-bc83-4fd5-ba21-caf7637709a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:45.748355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672cec6-bc83-4fd5-ba21-caf7637709a9 None admin] RBAC: Adding query filter params (role.id=b5e411fc96434df69c6cc1df951b3e6f, scope.project.id=ad5d2677b95b428a93e499f4ced1b86b, user.id=c9421b5ced544e1591c7ab5d6204d5f0) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:45.748672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672cec6-bc83-4fd5-ba21-caf7637709a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:45.750174 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672cec6-bc83-4fd5-ba21-caf7637709a9 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:45.756758 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 136/274] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:45 2022] GET /identity/v3/role_assignments?user.id=c9421b5ced544e1591c7ab5d6204d5f0&scope.project.id=ad5d2677b95b428a93e499f4ced1b86b&role.id=b5e411fc96434df69c6cc1df951b3e6f => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:45.757347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 139/275] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:45.784701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672cec6-bc83-4fd5-ba21-caf7637709a9 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:45.861890 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:45.898727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672cec6-bc83-4fd5-ba21-caf7637709a9 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:45.936897 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce343df-404e-4905-acb1-a55c69dd2284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:45.937552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce343df-404e-4905-acb1-a55c69dd2284 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:45.937836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce343df-404e-4905-acb1-a55c69dd2284 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:45.938115 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce343df-404e-4905-acb1-a55c69dd2284 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:45.939271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce343df-404e-4905-acb1-a55c69dd2284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:45.940785 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce343df-404e-4905-acb1-a55c69dd2284 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:45.942721 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 137/276] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:45.949505 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43924e-701f-4c48-a152-3ed668ec035d None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:46.047783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce343df-404e-4905-acb1-a55c69dd2284 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:46.094408 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43924e-701f-4c48-a152-3ed668ec035d None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:46.143809 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78add92-0daf-4297-a4b7-6b50ddf5e79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:46.144420 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78add92-0daf-4297-a4b7-6b50ddf5e79f None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:46.144752 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78add92-0daf-4297-a4b7-6b50ddf5e79f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:46.145149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78add92-0daf-4297-a4b7-6b50ddf5e79f None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:46.145815 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78add92-0daf-4297-a4b7-6b50ddf5e79f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:46.147267 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78add92-0daf-4297-a4b7-6b50ddf5e79f None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:46.177220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce343df-404e-4905-acb1-a55c69dd2284 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:46.210796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31bf410-a427-4264-99e5-30bfc46fce96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:46.211407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31bf410-a427-4264-99e5-30bfc46fce96 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:46.211688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31bf410-a427-4264-99e5-30bfc46fce96 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:46.211960 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31bf410-a427-4264-99e5-30bfc46fce96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:46.214644 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:46.223057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31bf410-a427-4264-99e5-30bfc46fce96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-d31bf410-a427-4264-99e5-30bfc46fce96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:46.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:46.229712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:46.230507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:46.230507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:46.230507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:46.230507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:46.230507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:46.230507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:46.230507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:46.230507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:46.230507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:46.230507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:46.230507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:46.230507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:46.230507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 138/277] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:00:46.231258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31bf410-a427-4264-99e5-30bfc46fce96 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:46.260678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 140/278] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:00:45 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:00:46.260678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78add92-0daf-4297-a4b7-6b50ddf5e79f None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:46.309151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:46.373050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31bf410-a427-4264-99e5-30bfc46fce96 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:46.382955 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78add92-0daf-4297-a4b7-6b50ddf5e79f None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:46.419630 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f0576-bb2c-400c-8c4a-68e4d3d2872f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:46.420258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f0576-bb2c-400c-8c4a-68e4d3d2872f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:46.420548 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f0576-bb2c-400c-8c4a-68e4d3d2872f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:46.420896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f0576-bb2c-400c-8c4a-68e4d3d2872f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:46.436459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21317f5d-a047-4207-898a-ac13b4e8feef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:46.436459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21317f5d-a047-4207-898a-ac13b4e8feef None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:46.436459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21317f5d-a047-4207-898a-ac13b4e8feef None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:46.436459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21317f5d-a047-4207-898a-ac13b4e8feef None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:46.437092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21317f5d-a047-4207-898a-ac13b4e8feef None admin] RBAC: Adding query filter params (name=member) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:46.437543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f0576-bb2c-400c-8c4a-68e4d3d2872f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:46.437647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21317f5d-a047-4207-898a-ac13b4e8feef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:46.438831 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21317f5d-a047-4207-898a-ac13b4e8feef None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-5a4f0576-bb2c-400c-8c4a-68e4d3d2872f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:46.444209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:46.445658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:46.446778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:46.446778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:46.446778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:46.446778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:46.446778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:46.446778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:46.446778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:46.446778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:46.446778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:46.446778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:46.446778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:46.446778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:46.446778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 139/279] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:46.452143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f0576-bb2c-400c-8c4a-68e4d3d2872f None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:46.457352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 141/280] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:46.459651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21317f5d-a047-4207-898a-ac13b4e8feef None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:46.517535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:46.586205 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f0576-bb2c-400c-8c4a-68e4d3d2872f None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:46.600194 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21317f5d-a047-4207-898a-ac13b4e8feef None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:46.606933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:46.627879 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945435a4-a1a6-4c3f-bc1d-8380137f0443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:46.628988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945435a4-a1a6-4c3f-bc1d-8380137f0443 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:46.629480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945435a4-a1a6-4c3f-bc1d-8380137f0443 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:46.629954 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945435a4-a1a6-4c3f-bc1d-8380137f0443 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:46.630808 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945435a4-a1a6-4c3f-bc1d-8380137f0443 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:46.631114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945435a4-a1a6-4c3f-bc1d-8380137f0443 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:46.633874 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945435a4-a1a6-4c3f-bc1d-8380137f0443 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:46.643297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f071e-9da2-46d2-bb7e-76dd8241caf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:46.643923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f071e-9da2-46d2-bb7e-76dd8241caf1 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:46.644201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f071e-9da2-46d2-bb7e-76dd8241caf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:46.644469 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f071e-9da2-46d2-bb7e-76dd8241caf1 None admin] PATH_INFO: `/v3/users/ef8a67d09e1b4aba97d2f51f9b58a912` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:46.646696 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 140/281] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:46.653016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945435a4-a1a6-4c3f-bc1d-8380137f0443 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:46.662827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f071e-9da2-46d2-bb7e-76dd8241caf1 None admin] RBAC: Authorizing `identity:get_user(user_id=ef8a67d09e1b4aba97d2f51f9b58a912)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:46.664439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f071e-9da2-46d2-bb7e-76dd8241caf1 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:46.666307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 142/282] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:46 2022] GET /identity/v3/users/ef8a67d09e1b4aba97d2f51f9b58a912 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:46.672716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f071e-9da2-46d2-bb7e-76dd8241caf1 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:46.826602 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945435a4-a1a6-4c3f-bc1d-8380137f0443 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:46.830525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f071e-9da2-46d2-bb7e-76dd8241caf1 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:46.860559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:46.885156 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d409c-9230-415a-b1d3-1ff7faba2d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:46.885767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d409c-9230-415a-b1d3-1ff7faba2d02 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:46.886051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d409c-9230-415a-b1d3-1ff7faba2d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:46.886319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d409c-9230-415a-b1d3-1ff7faba2d02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:46.887062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d409c-9230-415a-b1d3-1ff7faba2d02 None admin] RBAC: Adding query filter params (role.id=8b2ee9b11a934b67aa3fd68c497fbdf7, user.id=ef8a67d09e1b4aba97d2f51f9b58a912, scope.system=all) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:46.887372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d409c-9230-415a-b1d3-1ff7faba2d02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:46.889570 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d409c-9230-415a-b1d3-1ff7faba2d02 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:46.895013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc027031-cca3-47aa-806b-831ad1b21afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:46.895725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc027031-cca3-47aa-806b-831ad1b21afc None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:46.896009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc027031-cca3-47aa-806b-831ad1b21afc None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:46.896288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc027031-cca3-47aa-806b-831ad1b21afc None admin] PATH_INFO: `/v3/users/d9a9aab4d92f4d96a7046582cdc3661f` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:46.897761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc027031-cca3-47aa-806b-831ad1b21afc None admin] RBAC: Authorizing `identity:get_user(user_id=d9a9aab4d92f4d96a7046582cdc3661f)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:46.899411 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc027031-cca3-47aa-806b-831ad1b21afc None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:46.902699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 141/283] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:46 2022] GET /identity/v3/users/d9a9aab4d92f4d96a7046582cdc3661f => generated 297 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:46.905603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 143/284] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Aug 12 12:00:46 2022] GET /identity/v3/role_assignments?user.id=ef8a67d09e1b4aba97d2f51f9b58a912&scope.system=all&role.id=8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:46.907399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc027031-cca3-47aa-806b-831ad1b21afc None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:46.935246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e7d48-6164-4b3d-ae2e-d70a79ad1065 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:46.935603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e7d48-6164-4b3d-ae2e-d70a79ad1065 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:46.935984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e7d48-6164-4b3d-ae2e-d70a79ad1065 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:46.937711 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 144/285] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:00:46.942580 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31792f-76f2-4389-a18e-88dc5908903e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:46.942859 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31792f-76f2-4389-a18e-88dc5908903e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:46.943127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31792f-76f2-4389-a18e-88dc5908903e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:46.962558 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-9a31792f-76f2-4389-a18e-88dc5908903e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:46.963750 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9a31792f-76f2-4389-a18e-88dc5908903e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:46.987098 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:47.066674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc027031-cca3-47aa-806b-831ad1b21afc None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:47.079406 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 145/286] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:00:47.117629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d36248-15f0-4303-9cfb-d64d07c14a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:47.118557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d36248-15f0-4303-9cfb-d64d07c14a49 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:47.119007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d36248-15f0-4303-9cfb-d64d07c14a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:47.119449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d36248-15f0-4303-9cfb-d64d07c14a49 None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:47.120766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d36248-15f0-4303-9cfb-d64d07c14a49 None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:47.122405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d36248-15f0-4303-9cfb-d64d07c14a49 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:47.124549 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 142/287] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:46 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:47.130137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31792f-76f2-4389-a18e-88dc5908903e None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:47.231691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31792f-76f2-4389-a18e-88dc5908903e None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:47.266062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898ceaf-1068-41e2-a835-38aefaa78d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:47.267028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898ceaf-1068-41e2-a835-38aefaa78d3a None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:47.267476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898ceaf-1068-41e2-a835-38aefaa78d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:47.267917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898ceaf-1068-41e2-a835-38aefaa78d3a None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b/users/d9a9aab4d92f4d96a7046582cdc3661f/roles/bf2de8c80c5e4be79e1ea7adc3cbfc56` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:47.270029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898ceaf-1068-41e2-a835-38aefaa78d3a None admin] RBAC: Authorizing `identity:create_grant(project_id=ad5d2677b95b428a93e499f4ced1b86b, user_id=d9a9aab4d92f4d96a7046582cdc3661f, role_id=bf2de8c80c5e4be79e1ea7adc3cbfc56)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:47.271661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898ceaf-1068-41e2-a835-38aefaa78d3a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:47.341764 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 146/288] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:00:47 2022] PUT /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b/users/d9a9aab4d92f4d96a7046582cdc3661f/roles/bf2de8c80c5e4be79e1ea7adc3cbfc56 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:00:47.422718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d36248-15f0-4303-9cfb-d64d07c14a49 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:47.495813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:47.549453 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d36248-15f0-4303-9cfb-d64d07c14a49 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:47.587539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b3229-bc98-453e-92a8-3f0ac5014d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:47.588167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b3229-bc98-453e-92a8-3f0ac5014d48 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:47.588446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b3229-bc98-453e-92a8-3f0ac5014d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:47.588747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b3229-bc98-453e-92a8-3f0ac5014d48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:47.599105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b3229-bc98-453e-92a8-3f0ac5014d48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-bf4b3229-bc98-453e-92a8-3f0ac5014d48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:47.605621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:47.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:47.607300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:47.607300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:47.607300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:47.607300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:47.607300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:47.607300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:47.607300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:47.607300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:47.607300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:47.607300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:47.607300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:47.607300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:47.614456 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 143/289] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:47.615787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898ceaf-1068-41e2-a835-38aefaa78d3a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:47.636403 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:47.731468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898ceaf-1068-41e2-a835-38aefaa78d3a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:47.776347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabcb47-fcf7-4b1f-84ac-c2f0d52c6f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:47.777063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabcb47-fcf7-4b1f-84ac-c2f0d52c6f06 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:47.777346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabcb47-fcf7-4b1f-84ac-c2f0d52c6f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:47.777625 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabcb47-fcf7-4b1f-84ac-c2f0d52c6f06 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:47.778360 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabcb47-fcf7-4b1f-84ac-c2f0d52c6f06 None admin] RBAC: Adding query filter params (name=member) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:47.778670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabcb47-fcf7-4b1f-84ac-c2f0d52c6f06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:47.780110 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabcb47-fcf7-4b1f-84ac-c2f0d52c6f06 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:47.790660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 147/290] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:47.796263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b3229-bc98-453e-92a8-3f0ac5014d48 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:47.940369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b3229-bc98-453e-92a8-3f0ac5014d48 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:47.961740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:47.987629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ecb28-9566-406f-890d-883587f3a420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:47.988266 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ecb28-9566-406f-890d-883587f3a420 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:47.988569 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ecb28-9566-406f-890d-883587f3a420 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:47.993162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ecb28-9566-406f-890d-883587f3a420 None admin] PATH_INFO: `/v3/users/c2d7a3cf077348be9e4b9d1e0690439e` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:47.994344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ecb28-9566-406f-890d-883587f3a420 None admin] RBAC: Authorizing `identity:get_user(user_id=c2d7a3cf077348be9e4b9d1e0690439e)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:47.995919 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ecb28-9566-406f-890d-883587f3a420 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:47.998170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 144/291] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:47 2022] GET /identity/v3/users/c2d7a3cf077348be9e4b9d1e0690439e => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:48.003914 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabcb47-fcf7-4b1f-84ac-c2f0d52c6f06 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:48.163177 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:48.187812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabcb47-fcf7-4b1f-84ac-c2f0d52c6f06 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:48.237427 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cbd66-4904-43bd-ac2b-a53418278fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:48.237950 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cbd66-4904-43bd-ac2b-a53418278fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:48.238229 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cbd66-4904-43bd-ac2b-a53418278fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:48.238496 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9cbd66-4904-43bd-ac2b-a53418278fc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:48.327522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9cbd66-4904-43bd-ac2b-a53418278fc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:48.329054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9cbd66-4904-43bd-ac2b-a53418278fc6 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:48.440127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:48.464083 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 148/292] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:48.469596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ecb28-9566-406f-890d-883587f3a420 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:48.619803 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ecb28-9566-406f-890d-883587f3a420 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:48.664803 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f286ee-d379-4c17-a755-d41a8f8b3164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:48.665845 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f286ee-d379-4c17-a755-d41a8f8b3164 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:48.666332 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f286ee-d379-4c17-a755-d41a8f8b3164 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:48.666812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f286ee-d379-4c17-a755-d41a8f8b3164 None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:48.668162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f286ee-d379-4c17-a755-d41a8f8b3164 None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:48.669921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f286ee-d379-4c17-a755-d41a8f8b3164 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:48.672046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 145/293] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:48 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:48.677764 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cbd66-4904-43bd-ac2b-a53418278fc6 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:48.744382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a467-3fd2-4a07-961c-74a5eb43abca None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:48.744737 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a467-3fd2-4a07-961c-74a5eb43abca None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:48.744975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a467-3fd2-4a07-961c-74a5eb43abca None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:48.746582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 146/294] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:00:48.751514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dde2df-43df-4dfa-a5fc-31adc9a088fb None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:48.751798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dde2df-43df-4dfa-a5fc-31adc9a088fb None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:48.752075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dde2df-43df-4dfa-a5fc-31adc9a088fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:48.782426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-41dde2df-43df-4dfa-a5fc-31adc9a088fb None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:48.783661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-41dde2df-43df-4dfa-a5fc-31adc9a088fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:48.787678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9cbd66-4904-43bd-ac2b-a53418278fc6 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:48.830714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81b125-8692-452c-83af-4241f229100f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:48.832202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81b125-8692-452c-83af-4241f229100f None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:48.832679 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81b125-8692-452c-83af-4241f229100f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:48.833179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81b125-8692-452c-83af-4241f229100f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:48.834205 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81b125-8692-452c-83af-4241f229100f None admin] RBAC: Adding query filter params (role.id=8b2ee9b11a934b67aa3fd68c497fbdf7, scope.project.id=ad5d2677b95b428a93e499f4ced1b86b, user.id=c2d7a3cf077348be9e4b9d1e0690439e) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:48.834728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81b125-8692-452c-83af-4241f229100f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:48.836953 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81b125-8692-452c-83af-4241f229100f None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:48.843908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 149/295] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:48 2022] GET /identity/v3/role_assignments?user.id=c2d7a3cf077348be9e4b9d1e0690439e&scope.project.id=ad5d2677b95b428a93e499f4ced1b86b&role.id=8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:48.845404 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f7758-f5e7-449f-bde3-99ec929ec0c6 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:48.845905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f7758-f5e7-449f-bde3-99ec929ec0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:48.846371 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f7758-f5e7-449f-bde3-99ec929ec0c6 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:48.847810 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 150/296] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:48.856681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cb8f3-c8fd-4179-af28-68bdd4510c04 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:48.856681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cb8f3-c8fd-4179-af28-68bdd4510c04 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:48.856681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cb8f3-c8fd-4179-af28-68bdd4510c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:48.873361 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:48.880292 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-438cb8f3-c8fd-4179-af28-68bdd4510c04 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:48.882747 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-438cb8f3-c8fd-4179-af28-68bdd4510c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:48.908884 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 147/297] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:00:48.940801 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:48.986019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 151/298] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:49.234588 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dde2df-43df-4dfa-a5fc-31adc9a088fb None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:49.302594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cb8f3-c8fd-4179-af28-68bdd4510c04 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:49.355376 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dde2df-43df-4dfa-a5fc-31adc9a088fb None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:49.400728 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f3ef8-0f8f-4495-bf9d-f4504ad6e467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:49.401481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f3ef8-0f8f-4495-bf9d-f4504ad6e467 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:49.401761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f3ef8-0f8f-4495-bf9d-f4504ad6e467 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:49.402045 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f3ef8-0f8f-4495-bf9d-f4504ad6e467 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:49.403305 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f3ef8-0f8f-4495-bf9d-f4504ad6e467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:49.405254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f3ef8-0f8f-4495-bf9d-f4504ad6e467 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:49.407590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 148/299] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:49.413225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bfc1f-d684-4a87-8ad9-1c901d8a02c2 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:49.413548 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bfc1f-d684-4a87-8ad9-1c901d8a02c2 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:49.413762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bfc1f-d684-4a87-8ad9-1c901d8a02c2 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:49.415483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 149/300] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:49.416190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bfc1f-d684-4a87-8ad9-1c901d8a02c2 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:49.428338 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cb8f3-c8fd-4179-af28-68bdd4510c04 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:49.464038 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90eaee-d3ec-4d61-9409-fb962d34ad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:49.464848 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90eaee-d3ec-4d61-9409-fb962d34ad69 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:49.465131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90eaee-d3ec-4d61-9409-fb962d34ad69 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:49.465424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90eaee-d3ec-4d61-9409-fb962d34ad69 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:49.473073 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:49.476542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90eaee-d3ec-4d61-9409-fb962d34ad69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-ea90eaee-d3ec-4d61-9409-fb962d34ad69 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:49.481307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:49.482136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:49.482933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:49.482933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:49.482933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:49.482933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:49.482933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:49.482933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:49.482933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:49.482933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:49.482933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:49.482933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:49.482933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:49.482933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:49.482933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 152/301] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:00:49.483568 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478ff56-028b-458c-9cc6-b25f6ea03305 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:49.484244 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478ff56-028b-458c-9cc6-b25f6ea03305 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:49.484244 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478ff56-028b-458c-9cc6-b25f6ea03305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:49.502502 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-f478ff56-028b-458c-9cc6-b25f6ea03305 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:49.503598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f478ff56-028b-458c-9cc6-b25f6ea03305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:49.532056 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:49.536107 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bfc1f-d684-4a87-8ad9-1c901d8a02c2 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:49.571566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d620747-f50e-400c-98ff-fd52c0119caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:49.572185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d620747-f50e-400c-98ff-fd52c0119caf None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:49.572487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d620747-f50e-400c-98ff-fd52c0119caf None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:49.572755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d620747-f50e-400c-98ff-fd52c0119caf None admin] PATH_INFO: `/v3/groups` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:49.573381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d620747-f50e-400c-98ff-fd52c0119caf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:49.574810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d620747-f50e-400c-98ff-fd52c0119caf None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:49.578275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 153/302] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:49.578954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478ff56-028b-458c-9cc6-b25f6ea03305 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:49.653984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 150/303] 10.222.0.27 () {60 vars in 1134 bytes} [Fri Aug 12 12:00:49 2022] POST /identity/v3/groups => generated 229 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:00:49.696665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478ff56-028b-458c-9cc6-b25f6ea03305 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:49.736839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f2109-ca57-4ede-b426-1620ca67a70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:49.737456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f2109-ca57-4ede-b426-1620ca67a70e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:49.737736 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f2109-ca57-4ede-b426-1620ca67a70e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:49.738007 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f2109-ca57-4ede-b426-1620ca67a70e None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:49.738748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f2109-ca57-4ede-b426-1620ca67a70e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:49.739052 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f2109-ca57-4ede-b426-1620ca67a70e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:49.740485 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f2109-ca57-4ede-b426-1620ca67a70e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:49.750664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 154/304] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:49.756553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d620747-f50e-400c-98ff-fd52c0119caf None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:49.868406 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d620747-f50e-400c-98ff-fd52c0119caf None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:49.877760 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:49.879601 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f2109-ca57-4ede-b426-1620ca67a70e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:49.903822 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c659d-d2bc-4fb5-9a75-7082c657dfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:49.904458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c659d-d2bc-4fb5-9a75-7082c657dfd9 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:49.904848 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c659d-d2bc-4fb5-9a75-7082c657dfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:49.905163 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c659d-d2bc-4fb5-9a75-7082c657dfd9 None admin] PATH_INFO: `/v3/users/43115344775b4d1f80556897c437cc4e` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:49.920063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c659d-d2bc-4fb5-9a75-7082c657dfd9 None admin] RBAC: Authorizing `identity:get_user(user_id=43115344775b4d1f80556897c437cc4e)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:49.921535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c659d-d2bc-4fb5-9a75-7082c657dfd9 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:49.923345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 151/305] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:49 2022] GET /identity/v3/users/43115344775b4d1f80556897c437cc4e => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:49.929177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c659d-d2bc-4fb5-9a75-7082c657dfd9 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:49.929772 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:50.033555 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f2109-ca57-4ede-b426-1620ca67a70e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:50.053617 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c659d-d2bc-4fb5-9a75-7082c657dfd9 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:50.070306 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf68f80-9f4e-44d5-9053-fd6d6303403b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:50.070930 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf68f80-9f4e-44d5-9053-fd6d6303403b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:50.071207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf68f80-9f4e-44d5-9053-fd6d6303403b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:50.071473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf68f80-9f4e-44d5-9053-fd6d6303403b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:50.081656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf68f80-9f4e-44d5-9053-fd6d6303403b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-4bf68f80-9f4e-44d5-9053-fd6d6303403b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:50.087059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:50.088103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:50.089208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:50.089208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:50.089208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:50.089208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:50.089208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:50.089208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:50.089208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:50.089208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:50.089208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:50.089208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:50.089208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:50.089208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:50.089208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 155/306] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:50.090021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3f5ee-3b66-4c8f-b179-6061b661b32d None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:50.090021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3f5ee-3b66-4c8f-b179-6061b661b32d None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:50.090021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3f5ee-3b66-4c8f-b179-6061b661b32d None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:50.092683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 156/307] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:50.096133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3f5ee-3b66-4c8f-b179-6061b661b32d None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:50.098333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e538760-837b-47c3-94f8-d3719de12ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:50.098949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e538760-837b-47c3-94f8-d3719de12ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:50.099228 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e538760-837b-47c3-94f8-d3719de12ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:50.099505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e538760-837b-47c3-94f8-d3719de12ca0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:50.100253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e538760-837b-47c3-94f8-d3719de12ca0 None admin] RBAC: Adding query filter params (role.id=bf2de8c80c5e4be79e1ea7adc3cbfc56, user.id=43115344775b4d1f80556897c437cc4e, scope.system=all) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:50.100628 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e538760-837b-47c3-94f8-d3719de12ca0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:50.102132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e538760-837b-47c3-94f8-d3719de12ca0 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:50.107979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 152/308] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Aug 12 12:00:49 2022] GET /identity/v3/role_assignments?user.id=43115344775b4d1f80556897c437cc4e&scope.system=all&role.id=bf2de8c80c5e4be79e1ea7adc3cbfc56 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:50.109281 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedde915-70a5-4afa-8a2c-342f3fa678ba None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:50.109579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedde915-70a5-4afa-8a2c-342f3fa678ba None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:50.109846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedde915-70a5-4afa-8a2c-342f3fa678ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:50.136220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-fedde915-70a5-4afa-8a2c-342f3fa678ba None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:50.137781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-fedde915-70a5-4afa-8a2c-342f3fa678ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:50.166735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:50.217965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:50.234861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 153/309] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:50.242658 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3f5ee-3b66-4c8f-b179-6061b661b32d None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:50.281459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6535021-8bae-4f4c-9aab-a21806b15d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:50.283076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6535021-8bae-4f4c-9aab-a21806b15d14 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:50.283076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6535021-8bae-4f4c-9aab-a21806b15d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:50.283076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6535021-8bae-4f4c-9aab-a21806b15d14 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:50.283513 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6535021-8bae-4f4c-9aab-a21806b15d14 None admin] RBAC: Adding query filter params (name=member) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:50.283933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6535021-8bae-4f4c-9aab-a21806b15d14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:50.285345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6535021-8bae-4f4c-9aab-a21806b15d14 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:50.295786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 157/310] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:50.301074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedde915-70a5-4afa-8a2c-342f3fa678ba None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:50.409356 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedde915-70a5-4afa-8a2c-342f3fa678ba None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:50.445796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc5388-a674-440e-938d-8c5c95f618d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:50.446592 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc5388-a674-440e-938d-8c5c95f618d0 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:50.446965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc5388-a674-440e-938d-8c5c95f618d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:50.447328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc5388-a674-440e-938d-8c5c95f618d0 None admin] PATH_INFO: `/v3/users/ef8a67d09e1b4aba97d2f51f9b58a912` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:50.448543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc5388-a674-440e-938d-8c5c95f618d0 None admin] RBAC: Authorizing `identity:get_user(user_id=ef8a67d09e1b4aba97d2f51f9b58a912)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:50.450110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc5388-a674-440e-938d-8c5c95f618d0 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:50.452083 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 154/311] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:50 2022] GET /identity/v3/users/ef8a67d09e1b4aba97d2f51f9b58a912 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:50.457570 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6535021-8bae-4f4c-9aab-a21806b15d14 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:50.514929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc5388-a674-440e-938d-8c5c95f618d0 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:50.545260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:50.572677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6535021-8bae-4f4c-9aab-a21806b15d14 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:50.611606 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440450e-e36b-4856-b6cf-fef53b9e1572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:50.612248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440450e-e36b-4856-b6cf-fef53b9e1572 None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:50.612529 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440450e-e36b-4856-b6cf-fef53b9e1572 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:50.612980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440450e-e36b-4856-b6cf-fef53b9e1572 None admin] PATH_INFO: `/v3/system/users/ef8a67d09e1b4aba97d2f51f9b58a912/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:50.614616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440450e-e36b-4856-b6cf-fef53b9e1572 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ef8a67d09e1b4aba97d2f51f9b58a912, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:50.616499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440450e-e36b-4856-b6cf-fef53b9e1572 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:50.630275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc5388-a674-440e-938d-8c5c95f618d0 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:50.667364 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e00f2-f489-4459-b72f-6f9b80eaca8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:50.668040 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e00f2-f489-4459-b72f-6f9b80eaca8b None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:50.668322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e00f2-f489-4459-b72f-6f9b80eaca8b None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:50.668625 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e00f2-f489-4459-b72f-6f9b80eaca8b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:50.679434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e00f2-f489-4459-b72f-6f9b80eaca8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-5a3e00f2-f489-4459-b72f-6f9b80eaca8b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:50.684435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:50.685486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:50.686441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:50.686441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:50.686441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:50.686441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:50.686441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:50.686441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:50.686441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:50.686441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:50.686441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:50.686441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:50.686441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:50.686441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:50.686441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 155/312] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:00:50.693196 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e00f2-f489-4459-b72f-6f9b80eaca8b None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:50.712326 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 158/313] 10.222.0.27 () {58 vars in 1334 bytes} [Fri Aug 12 12:00:50 2022] PUT /identity/v3/system/users/ef8a67d09e1b4aba97d2f51f9b58a912/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:00:50.808481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:50.822158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e00f2-f489-4459-b72f-6f9b80eaca8b None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:50.860397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94df50-1eaa-4e9f-a5dc-a92f61d25e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:50.861170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94df50-1eaa-4e9f-a5dc-a92f61d25e5b None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:50.861548 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94df50-1eaa-4e9f-a5dc-a92f61d25e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:50.861933 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94df50-1eaa-4e9f-a5dc-a92f61d25e5b None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:50.862697 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94df50-1eaa-4e9f-a5dc-a92f61d25e5b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:50.863097 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94df50-1eaa-4e9f-a5dc-a92f61d25e5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:50.864561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94df50-1eaa-4e9f-a5dc-a92f61d25e5b None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:50.873340 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 156/314] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:50.878951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440450e-e36b-4856-b6cf-fef53b9e1572 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:50.893474 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:50.985572 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440450e-e36b-4856-b6cf-fef53b9e1572 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:51.019527 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d28ae-4b66-4c2a-beb1-bbbe136af5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:51.020146 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d28ae-4b66-4c2a-beb1-bbbe136af5ac None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:51.020424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d28ae-4b66-4c2a-beb1-bbbe136af5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:51.020719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d28ae-4b66-4c2a-beb1-bbbe136af5ac None admin] PATH_INFO: `/v3/users/d9a9aab4d92f4d96a7046582cdc3661f` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:51.021845 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d28ae-4b66-4c2a-beb1-bbbe136af5ac None admin] RBAC: Authorizing `identity:get_user(user_id=d9a9aab4d92f4d96a7046582cdc3661f)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:51.023982 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d28ae-4b66-4c2a-beb1-bbbe136af5ac None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:51.025895 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 159/315] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:50 2022] GET /identity/v3/users/d9a9aab4d92f4d96a7046582cdc3661f => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:51.031837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94df50-1eaa-4e9f-a5dc-a92f61d25e5b None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:51.139191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94df50-1eaa-4e9f-a5dc-a92f61d25e5b None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:51.174931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e23fd-5ceb-4166-86a4-9700e71ff593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:51.175921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e23fd-5ceb-4166-86a4-9700e71ff593 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:51.176399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e23fd-5ceb-4166-86a4-9700e71ff593 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:51.176890 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e23fd-5ceb-4166-86a4-9700e71ff593 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:51.198932 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:51.257719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0e23fd-5ceb-4166-86a4-9700e71ff593 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:51.259575 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0e23fd-5ceb-4166-86a4-9700e71ff593 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:51.369896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 157/316] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:51.375449 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d28ae-4b66-4c2a-beb1-bbbe136af5ac None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:51.455379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:51.477368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d28ae-4b66-4c2a-beb1-bbbe136af5ac None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:51.511553 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13845c4d-5804-4c09-b656-6fb1f57dfe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:51.512178 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13845c4d-5804-4c09-b656-6fb1f57dfe8e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:51.512455 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13845c4d-5804-4c09-b656-6fb1f57dfe8e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:51.512756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13845c4d-5804-4c09-b656-6fb1f57dfe8e None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:51.513972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13845c4d-5804-4c09-b656-6fb1f57dfe8e None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:51.515387 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13845c4d-5804-4c09-b656-6fb1f57dfe8e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:51.517305 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 160/317] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:51 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:51.523098 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e23fd-5ceb-4166-86a4-9700e71ff593 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:51.576927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:51.629271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e23fd-5ceb-4166-86a4-9700e71ff593 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:51.666018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecd479-b530-4f8b-b0c1-70064de3aa8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:51.666620 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecd479-b530-4f8b-b0c1-70064de3aa8e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:51.666896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecd479-b530-4f8b-b0c1-70064de3aa8e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:51.667220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecd479-b530-4f8b-b0c1-70064de3aa8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:51.667949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecd479-b530-4f8b-b0c1-70064de3aa8e None admin] RBAC: Adding query filter params (role.id=bf2de8c80c5e4be79e1ea7adc3cbfc56, scope.project.id=ad5d2677b95b428a93e499f4ced1b86b, user.id=d9a9aab4d92f4d96a7046582cdc3661f) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:51.668253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecd479-b530-4f8b-b0c1-70064de3aa8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:51.669868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecd479-b530-4f8b-b0c1-70064de3aa8e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:51.676047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 158/318] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:00:51 2022] GET /identity/v3/role_assignments?user.id=d9a9aab4d92f4d96a7046582cdc3661f&scope.project.id=ad5d2677b95b428a93e499f4ced1b86b&role.id=bf2de8c80c5e4be79e1ea7adc3cbfc56 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:51.796587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5835b3-fab3-466b-883d-e67c505321b5 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:51.797346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5835b3-fab3-466b-883d-e67c505321b5 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:51.797685 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5835b3-fab3-466b-883d-e67c505321b5 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:51.799253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 161/319] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:51.814768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c3c28-9f8d-402a-ac08-d6db6bc3ca21 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:51.815064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c3c28-9f8d-402a-ac08-d6db6bc3ca21 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:51.815343 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c3c28-9f8d-402a-ac08-d6db6bc3ca21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:51.864666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-571c3c28-9f8d-402a-ac08-d6db6bc3ca21 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:51.864666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-571c3c28-9f8d-402a-ac08-d6db6bc3ca21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:51.952280 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 159/320] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:52.149436 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f610e-bd4b-47b2-aeab-050d6e3f2561 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:52.149933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f610e-bd4b-47b2-aeab-050d6e3f2561 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:52.150312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f610e-bd4b-47b2-aeab-050d6e3f2561 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:52.151755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 162/321] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:52.157159 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e9d2a-c5a6-4b01-ba53-8970b56a5efc None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:52.157559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e9d2a-c5a6-4b01-ba53-8970b56a5efc None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:52.157933 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e9d2a-c5a6-4b01-ba53-8970b56a5efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:52.176651 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-fc1e9d2a-c5a6-4b01-ba53-8970b56a5efc None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:52.178033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-fc1e9d2a-c5a6-4b01-ba53-8970b56a5efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:52.230089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:52.248237 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f610e-bd4b-47b2-aeab-050d6e3f2561 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:52.257296 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 160/322] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:52.353828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f610e-bd4b-47b2-aeab-050d6e3f2561 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:52.388140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159b66c-1b56-489d-a52f-1167c95ea05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:52.388795 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159b66c-1b56-489d-a52f-1167c95ea05a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:52.389077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159b66c-1b56-489d-a52f-1167c95ea05a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:52.389347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159b66c-1b56-489d-a52f-1167c95ea05a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:52.390431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159b66c-1b56-489d-a52f-1167c95ea05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:52.391868 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159b66c-1b56-489d-a52f-1167c95ea05a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:52.393681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 163/323] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:00:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:52.399862 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e9d2a-c5a6-4b01-ba53-8970b56a5efc None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:52.501779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e9d2a-c5a6-4b01-ba53-8970b56a5efc None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:52.523702 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159b66c-1b56-489d-a52f-1167c95ea05a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:52.537368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8267d6-292b-4722-a5d4-39dffc59de38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:52.540979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8267d6-292b-4722-a5d4-39dffc59de38 None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:52.541262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8267d6-292b-4722-a5d4-39dffc59de38 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:52.541532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8267d6-292b-4722-a5d4-39dffc59de38 None admin] PATH_INFO: `/v3/groups` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:52.542172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8267d6-292b-4722-a5d4-39dffc59de38 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:52.543623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8267d6-292b-4722-a5d4-39dffc59de38 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:52.544174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:52.628537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159b66c-1b56-489d-a52f-1167c95ea05a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:52.659128 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 161/324] 10.222.0.27 () {60 vars in 1134 bytes} [Fri Aug 12 12:00:52 2022] POST /identity/v3/groups => generated 226 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:00:52.667898 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5f98f-7db0-4c18-8410-c5be883a65b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:52.668807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5f98f-7db0-4c18-8410-c5be883a65b9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:52.669095 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5f98f-7db0-4c18-8410-c5be883a65b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:52.669369 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5f98f-7db0-4c18-8410-c5be883a65b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:52.682483 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5f98f-7db0-4c18-8410-c5be883a65b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-07a5f98f-7db0-4c18-8410-c5be883a65b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:52.687952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:52.688989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:52.693575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:52.693575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:52.693575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:52.693575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:52.693575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:52.693575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:52.693575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:52.693575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:52.693575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:52.693575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:52.693575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:52.693575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:52.693575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 164/325] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:00:52.695793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8267d6-292b-4722-a5d4-39dffc59de38 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:52.736012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:52.815998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8267d6-292b-4722-a5d4-39dffc59de38 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:52.851632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a90f0d-29e5-49c8-97b3-a0b6259d1a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:52.852238 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a90f0d-29e5-49c8-97b3-a0b6259d1a2c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:52.852519 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a90f0d-29e5-49c8-97b3-a0b6259d1a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:52.852834 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a90f0d-29e5-49c8-97b3-a0b6259d1a2c None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:52.853503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a90f0d-29e5-49c8-97b3-a0b6259d1a2c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:52.853810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a90f0d-29e5-49c8-97b3-a0b6259d1a2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:52.855185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a90f0d-29e5-49c8-97b3-a0b6259d1a2c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:52.858884 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e763f-d356-41fa-aa1a-9df71c543e85 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:52.859174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e763f-d356-41fa-aa1a-9df71c543e85 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:52.859447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e763f-d356-41fa-aa1a-9df71c543e85 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:52.860949 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 165/326] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:00:52.863807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 162/327] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:52.866751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b55fa-840f-4bfe-959f-9a28fcd09be6 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:52.867042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b55fa-840f-4bfe-959f-9a28fcd09be6 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:52.867318 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b55fa-840f-4bfe-959f-9a28fcd09be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:52.869208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a90f0d-29e5-49c8-97b3-a0b6259d1a2c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:52.887937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-529b55fa-840f-4bfe-959f-9a28fcd09be6 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:52.889493 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-529b55fa-840f-4bfe-959f-9a28fcd09be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:52.983940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 166/328] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:52.992090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:53.001843 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a90f0d-29e5-49c8-97b3-a0b6259d1a2c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:53.040604 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817676e-323e-4176-a24b-16283cb2bc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:53.041266 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817676e-323e-4176-a24b-16283cb2bc65 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:53.041545 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817676e-323e-4176-a24b-16283cb2bc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:53.041818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817676e-323e-4176-a24b-16283cb2bc65 None admin] PATH_INFO: `/v3/users/43115344775b4d1f80556897c437cc4e` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:53.042962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3817676e-323e-4176-a24b-16283cb2bc65 None admin] RBAC: Authorizing `identity:get_user(user_id=43115344775b4d1f80556897c437cc4e)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:53.044403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3817676e-323e-4176-a24b-16283cb2bc65 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:53.046163 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 163/329] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:52 2022] GET /identity/v3/users/43115344775b4d1f80556897c437cc4e => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:53.051871 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b55fa-840f-4bfe-959f-9a28fcd09be6 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:53.057467 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:53.169329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b55fa-840f-4bfe-959f-9a28fcd09be6 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:53.207948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf54dd-d217-44b0-a2d1-b36344d28cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:53.208602 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf54dd-d217-44b0-a2d1-b36344d28cd7 None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:53.208910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf54dd-d217-44b0-a2d1-b36344d28cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:53.209183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf54dd-d217-44b0-a2d1-b36344d28cd7 None admin] PATH_INFO: `/v3/system/users/43115344775b4d1f80556897c437cc4e/roles/bf2de8c80c5e4be79e1ea7adc3cbfc56` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:53.211325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf54dd-d217-44b0-a2d1-b36344d28cd7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=43115344775b4d1f80556897c437cc4e, role_id=bf2de8c80c5e4be79e1ea7adc3cbfc56)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:53.212750 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf54dd-d217-44b0-a2d1-b36344d28cd7 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:53.268960 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817676e-323e-4176-a24b-16283cb2bc65 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:53.290216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 167/330] 10.222.0.27 () {58 vars in 1334 bytes} [Fri Aug 12 12:00:53 2022] PUT /identity/v3/system/users/43115344775b4d1f80556897c437cc4e/roles/bf2de8c80c5e4be79e1ea7adc3cbfc56 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:00:53.390613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817676e-323e-4176-a24b-16283cb2bc65 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:53.432253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d9d6b-256a-4612-93ae-f14abb5a6750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:53.433089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d9d6b-256a-4612-93ae-f14abb5a6750 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:53.433435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d9d6b-256a-4612-93ae-f14abb5a6750 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:53.433707 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d9d6b-256a-4612-93ae-f14abb5a6750 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:53.448543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d9d6b-256a-4612-93ae-f14abb5a6750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-f30d9d6b-256a-4612-93ae-f14abb5a6750 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:53.454156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:53.455008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:53.455795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:53.455795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:53.455795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:53.455795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:53.455795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:53.455795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:53.455795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:53.455795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:53.455795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:53.455795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:53.455795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:53.455795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:53.455795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 164/331] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:53.461748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf54dd-d217-44b0-a2d1-b36344d28cd7 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:53.531474 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:53.566045 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf54dd-d217-44b0-a2d1-b36344d28cd7 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:53.608275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d813c-b2fd-4af5-90b8-c140246b5121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:53.608916 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d813c-b2fd-4af5-90b8-c140246b5121 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:53.609192 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d813c-b2fd-4af5-90b8-c140246b5121 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:53.609460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d813c-b2fd-4af5-90b8-c140246b5121 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:53.610138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d813c-b2fd-4af5-90b8-c140246b5121 None admin] RBAC: Adding query filter params (name=member) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:53.610431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d813c-b2fd-4af5-90b8-c140246b5121 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:53.611807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d813c-b2fd-4af5-90b8-c140246b5121 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:53.624745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 168/332] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:53.630236 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d9d6b-256a-4612-93ae-f14abb5a6750 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:53.673881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:53.762035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d9d6b-256a-4612-93ae-f14abb5a6750 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:53.800685 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cd4af-8f8f-409f-851f-87242418eec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:53.801278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cd4af-8f8f-409f-851f-87242418eec0 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:53.801559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cd4af-8f8f-409f-851f-87242418eec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:53.801883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cd4af-8f8f-409f-851f-87242418eec0 None admin] PATH_INFO: `/v3/users/ef8a67d09e1b4aba97d2f51f9b58a912` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:53.803105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cd4af-8f8f-409f-851f-87242418eec0 None admin] RBAC: Authorizing `identity:get_user(user_id=ef8a67d09e1b4aba97d2f51f9b58a912)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:53.804794 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cd4af-8f8f-409f-851f-87242418eec0 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:53.807014 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 165/333] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:53 2022] GET /identity/v3/users/ef8a67d09e1b4aba97d2f51f9b58a912 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:53.813335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d813c-b2fd-4af5-90b8-c140246b5121 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:53.936502 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d813c-b2fd-4af5-90b8-c140246b5121 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:53.981015 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:53.985399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c82b9-20b3-4220-84ba-92d1d594b0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:53.986020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c82b9-20b3-4220-84ba-92d1d594b0ff None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:53.986309 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c82b9-20b3-4220-84ba-92d1d594b0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:53.986591 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c82b9-20b3-4220-84ba-92d1d594b0ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:53.987340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c82b9-20b3-4220-84ba-92d1d594b0ff None admin] RBAC: Adding query filter params (role.id=8b2ee9b11a934b67aa3fd68c497fbdf7, user.id=ef8a67d09e1b4aba97d2f51f9b58a912, scope.system=all) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:53.987656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c82b9-20b3-4220-84ba-92d1d594b0ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:53.994085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c82b9-20b3-4220-84ba-92d1d594b0ff None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:54.010788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 169/334] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Aug 12 12:00:53 2022] GET /identity/v3/role_assignments?user.id=ef8a67d09e1b4aba97d2f51f9b58a912&scope.system=all&role.id=8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:54.993903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea13cb-d8fe-4e3b-b04e-cbe52067bb2f None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:54.994432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea13cb-d8fe-4e3b-b04e-cbe52067bb2f None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:54.994432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea13cb-d8fe-4e3b-b04e-cbe52067bb2f None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:54.995817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 166/335] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:55.012173 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d2305-41c2-44bb-8fa3-3b439ef08d63 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:55.017909 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d2305-41c2-44bb-8fa3-3b439ef08d63 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:55.017909 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d2305-41c2-44bb-8fa3-3b439ef08d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:55.035303 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-b33d2305-41c2-44bb-8fa3-3b439ef08d63 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:55.036548 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b33d2305-41c2-44bb-8fa3-3b439ef08d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:55.137573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 170/336] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:55.439009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e9230-dbbc-4ab0-8d6b-2d90f6ccdb2e None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:55.439009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e9230-dbbc-4ab0-8d6b-2d90f6ccdb2e None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:55.439296 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e9230-dbbc-4ab0-8d6b-2d90f6ccdb2e None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:55.440988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 167/337] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:55.446819 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb01733-2751-48d8-9baa-0ff7cd8688c8 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:55.447110 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb01733-2751-48d8-9baa-0ff7cd8688c8 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:55.447394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb01733-2751-48d8-9baa-0ff7cd8688c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:55.472793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-3eb01733-2751-48d8-9baa-0ff7cd8688c8 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:55.474007 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-3eb01733-2751-48d8-9baa-0ff7cd8688c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:55.564471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 171/338] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:55.604912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d365e-1a9e-4b0b-bd01-f9c555db9202 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:55.605300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d365e-1a9e-4b0b-bd01-f9c555db9202 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:55.605459 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d365e-1a9e-4b0b-bd01-f9c555db9202 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:55.607056 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 168/339] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:55.616441 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6904259-1cee-4e97-bb72-ae9181c923e9 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:55.616441 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6904259-1cee-4e97-bb72-ae9181c923e9 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:55.616441 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6904259-1cee-4e97-bb72-ae9181c923e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:55.649378 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-a6904259-1cee-4e97-bb72-ae9181c923e9 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:55.650597 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a6904259-1cee-4e97-bb72-ae9181c923e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:55.721174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:55.748268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 172/340] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:00:55.792517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d365e-1a9e-4b0b-bd01-f9c555db9202 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:55.925055 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:55.928149 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d365e-1a9e-4b0b-bd01-f9c555db9202 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:55.975765 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c1f00-3545-4d04-b740-32a0d897e8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:55.976358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c1f00-3545-4d04-b740-32a0d897e8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:55.977580 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c1f00-3545-4d04-b740-32a0d897e8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:55.977580 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c1f00-3545-4d04-b740-32a0d897e8d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:55.995971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c1f00-3545-4d04-b740-32a0d897e8d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-fb5c1f00-3545-4d04-b740-32a0d897e8d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:56.000880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:56.002143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:56.004227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:56.004227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:56.004227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:56.004227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:56.004227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:56.004227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:56.004227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:56.004227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:56.004227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:56.004227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:56.004227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:56.004227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:56.004227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 169/341] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:56.015139 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6904259-1cee-4e97-bb72-ae9181c923e9 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:56.144294 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6904259-1cee-4e97-bb72-ae9181c923e9 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:56.177082 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c1f00-3545-4d04-b740-32a0d897e8d9 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:56.202691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468148e-63ec-4a6b-9c47-6a326eb2622c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:56.202691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468148e-63ec-4a6b-9c47-6a326eb2622c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:56.202691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468148e-63ec-4a6b-9c47-6a326eb2622c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:56.203041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468148e-63ec-4a6b-9c47-6a326eb2622c None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:56.203598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468148e-63ec-4a6b-9c47-6a326eb2622c None admin] RBAC: Adding query filter params (name=member) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:56.203903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468148e-63ec-4a6b-9c47-6a326eb2622c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:56.205615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468148e-63ec-4a6b-9c47-6a326eb2622c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:56.219171 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 173/342] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:56.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468148e-63ec-4a6b-9c47-6a326eb2622c None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:56.336088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:56.382345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c1f00-3545-4d04-b740-32a0d897e8d9 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:56.425579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf6ee2-637d-4cdb-ab7e-d4ce4272ef68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:56.426216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf6ee2-637d-4cdb-ab7e-d4ce4272ef68 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:56.426499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf6ee2-637d-4cdb-ab7e-d4ce4272ef68 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:56.426774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf6ee2-637d-4cdb-ab7e-d4ce4272ef68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:56.438619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf6ee2-637d-4cdb-ab7e-d4ce4272ef68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-1fcf6ee2-637d-4cdb-ab7e-d4ce4272ef68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:56.443503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:56.444969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:56.445774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:56.445774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:56.445774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:56.445774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:56.445774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:56.445774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:56.445774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:56.445774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:56.445774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:56.445774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:56.445774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:56.445774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:56.445774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 170/343] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:00:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:00:56.446414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf6ee2-637d-4cdb-ab7e-d4ce4272ef68 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:56.461208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468148e-63ec-4a6b-9c47-6a326eb2622c None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:56.464423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:56.529611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc1401-1a64-4c0e-b687-1501e63a951f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:56.530222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc1401-1a64-4c0e-b687-1501e63a951f None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:56.530497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc1401-1a64-4c0e-b687-1501e63a951f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:56.530765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc1401-1a64-4c0e-b687-1501e63a951f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:56.615729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf6ee2-637d-4cdb-ab7e-d4ce4272ef68 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:56.660927 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc1401-1a64-4c0e-b687-1501e63a951f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:56.662756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc1401-1a64-4c0e-b687-1501e63a951f None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:56.682981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5d965-10af-4438-9cf1-a0cc987bf44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:56.683604 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5d965-10af-4438-9cf1-a0cc987bf44a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:56.683883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5d965-10af-4438-9cf1-a0cc987bf44a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:56.684153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5d965-10af-4438-9cf1-a0cc987bf44a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:56.704457 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5d965-10af-4438-9cf1-a0cc987bf44a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-66f5d965-10af-4438-9cf1-a0cc987bf44a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:56.711395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:56.712285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:56.713195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:56.713195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:56.713195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:56.713195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:56.713195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:56.713195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:56.713195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:56.713195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:56.713195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:56.713195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:56.713195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:00:56.713195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:56.713806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 171/344] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:56.714518 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5d965-10af-4438-9cf1-a0cc987bf44a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:56.745112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:56.837010 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:56.840087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 174/345] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:56.840087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc1401-1a64-4c0e-b687-1501e63a951f None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:56.869709 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5d965-10af-4438-9cf1-a0cc987bf44a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:56.921913 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368c4cd-c7a5-42a0-8bcd-a39a82a1a3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:56.922527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368c4cd-c7a5-42a0-8bcd-a39a82a1a3bf None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:56.922810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368c4cd-c7a5-42a0-8bcd-a39a82a1a3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:56.923084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368c4cd-c7a5-42a0-8bcd-a39a82a1a3bf None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:56.923809 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368c4cd-c7a5-42a0-8bcd-a39a82a1a3bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:56.924089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368c4cd-c7a5-42a0-8bcd-a39a82a1a3bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:56.925526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368c4cd-c7a5-42a0-8bcd-a39a82a1a3bf None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:56.940762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 172/346] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:00:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:56.941885 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc59073-dde8-4d54-8aac-e4077e0fa25d None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:56.942173 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc59073-dde8-4d54-8aac-e4077e0fa25d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:56.942452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc59073-dde8-4d54-8aac-e4077e0fa25d None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:56.943688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 173/347] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:56.948736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc59073-dde8-4d54-8aac-e4077e0fa25d None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:56.983191 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc1401-1a64-4c0e-b687-1501e63a951f None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:57.023777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d692bcb-8a3f-46b3-91e0-a0795a6e6b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:57.024390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d692bcb-8a3f-46b3-91e0-a0795a6e6b42 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:57.024767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d692bcb-8a3f-46b3-91e0-a0795a6e6b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:57.025049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d692bcb-8a3f-46b3-91e0-a0795a6e6b42 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:57.025721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d692bcb-8a3f-46b3-91e0-a0795a6e6b42 None admin] RBAC: Adding query filter params (name=member) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:57.026024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d692bcb-8a3f-46b3-91e0-a0795a6e6b42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:57.028817 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d692bcb-8a3f-46b3-91e0-a0795a6e6b42 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:57.038150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 175/348] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:57.039629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe14a8e-c36d-4527-b14d-0a08018999eb None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:57.039915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe14a8e-c36d-4527-b14d-0a08018999eb None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:57.040194 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe14a8e-c36d-4527-b14d-0a08018999eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:57.049133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:57.063376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-9fe14a8e-c36d-4527-b14d-0a08018999eb None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:57.064603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9fe14a8e-c36d-4527-b14d-0a08018999eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:57.075119 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc59073-dde8-4d54-8aac-e4077e0fa25d None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:57.112120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf009c1-c198-4e2c-ac46-23bb3cbe778a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:57.113076 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf009c1-c198-4e2c-ac46-23bb3cbe778a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:57.113527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf009c1-c198-4e2c-ac46-23bb3cbe778a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:57.113968 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf009c1-c198-4e2c-ac46-23bb3cbe778a None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:57.115284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf009c1-c198-4e2c-ac46-23bb3cbe778a None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:57.117423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf009c1-c198-4e2c-ac46-23bb3cbe778a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:57.119522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 174/349] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:56 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:57.120576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf009c1-c198-4e2c-ac46-23bb3cbe778a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:57.136126 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:57.145266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 176/350] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:57.146201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe14a8e-c36d-4527-b14d-0a08018999eb None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:57.229721 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf009c1-c198-4e2c-ac46-23bb3cbe778a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:57.259683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe14a8e-c36d-4527-b14d-0a08018999eb None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:57.266898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd287f98-5925-4477-b6ea-d0192a445f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:57.267778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd287f98-5925-4477-b6ea-d0192a445f46 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:57.268228 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd287f98-5925-4477-b6ea-d0192a445f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:57.268692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd287f98-5925-4477-b6ea-d0192a445f46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:57.293500 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:57.299274 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835b440-49f0-4a9e-b865-1cd64b9af5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:57.318829 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835b440-49f0-4a9e-b865-1cd64b9af5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:57.318829 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835b440-49f0-4a9e-b865-1cd64b9af5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:57.318829 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835b440-49f0-4a9e-b865-1cd64b9af5c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:57.344800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd287f98-5925-4477-b6ea-d0192a445f46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:57.346793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd287f98-5925-4477-b6ea-d0192a445f46 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:57.375521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:57.383900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835b440-49f0-4a9e-b865-1cd64b9af5c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:57.383900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835b440-49f0-4a9e-b865-1cd64b9af5c5 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:57.471638 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 175/351] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:57.472834 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd287f98-5925-4477-b6ea-d0192a445f46 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:57.496121 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 177/352] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:00:57.497837 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835b440-49f0-4a9e-b865-1cd64b9af5c5 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:57.530436 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:57.587898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd287f98-5925-4477-b6ea-d0192a445f46 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:57.608814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835b440-49f0-4a9e-b865-1cd64b9af5c5 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:57.626018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dfacc-bfee-4758-a2c5-7212a68d80a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:57.627063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dfacc-bfee-4758-a2c5-7212a68d80a4 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:57.627839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dfacc-bfee-4758-a2c5-7212a68d80a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:57.628511 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dfacc-bfee-4758-a2c5-7212a68d80a4 None admin] PATH_INFO: `/v3/groups/8b050f7eb7bb48f4bc556655ec209b61` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:57.640862 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:57.641374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dfacc-bfee-4758-a2c5-7212a68d80a4 None admin] RBAC: Authorizing `identity:get_group(group_id=8b050f7eb7bb48f4bc556655ec209b61)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:57.644023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dfacc-bfee-4758-a2c5-7212a68d80a4 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:57.649491 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 176/353] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 12 12:00:57 2022] GET /identity/v3/groups/8b050f7eb7bb48f4bc556655ec209b61 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:57.656715 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dfacc-bfee-4758-a2c5-7212a68d80a4 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:57.665062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583570b9-02e8-4206-8ef3-de83137441fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:57.665654 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583570b9-02e8-4206-8ef3-de83137441fa None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:57.665939 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583570b9-02e8-4206-8ef3-de83137441fa None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:57.666217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583570b9-02e8-4206-8ef3-de83137441fa None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:57.669376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583570b9-02e8-4206-8ef3-de83137441fa None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:57.671153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583570b9-02e8-4206-8ef3-de83137441fa None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:57.674149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 178/354] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:57 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:57.675832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9f04d-dc41-4b3d-86bb-6d3247b587b9 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:57.676573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9f04d-dc41-4b3d-86bb-6d3247b587b9 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:57.682790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9f04d-dc41-4b3d-86bb-6d3247b587b9 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:57.684846 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 179/355] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:57.685627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9f04d-dc41-4b3d-86bb-6d3247b587b9 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:57.791544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dfacc-bfee-4758-a2c5-7212a68d80a4 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:57.803112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:57.809593 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9f04d-dc41-4b3d-86bb-6d3247b587b9 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:57.835548 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73102cc-d10e-48dc-98a8-31f80eec8588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:57.836599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73102cc-d10e-48dc-98a8-31f80eec8588 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:57.836971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73102cc-d10e-48dc-98a8-31f80eec8588 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:57.837243 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73102cc-d10e-48dc-98a8-31f80eec8588 None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:57.838395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73102cc-d10e-48dc-98a8-31f80eec8588 None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:57.839830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73102cc-d10e-48dc-98a8-31f80eec8588 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:57.842000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 177/356] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:57 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:57.842973 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6373ad-d74a-435f-898d-a472f6cf5e7f None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:57.843252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6373ad-d74a-435f-898d-a472f6cf5e7f None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:57.843522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6373ad-d74a-435f-898d-a472f6cf5e7f None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:57.844944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 178/357] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:00:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:00:57.845417 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6373ad-d74a-435f-898d-a472f6cf5e7f None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:57.852322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cb1c3-2e32-4a4b-a0af-48f6e7ed763c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:57.853168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cb1c3-2e32-4a4b-a0af-48f6e7ed763c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:57.853274 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cb1c3-2e32-4a4b-a0af-48f6e7ed763c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:57.853526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cb1c3-2e32-4a4b-a0af-48f6e7ed763c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:57.854274 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cb1c3-2e32-4a4b-a0af-48f6e7ed763c None admin] RBAC: Adding query filter params (group.id=8b050f7eb7bb48f4bc556655ec209b61, role.id=8b2ee9b11a934b67aa3fd68c497fbdf7, scope.project.id=4fe97b975d834d83a6733a0b2591bba4) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:57.854588 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cb1c3-2e32-4a4b-a0af-48f6e7ed763c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:57.856523 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cb1c3-2e32-4a4b-a0af-48f6e7ed763c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:57.866722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 180/358] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 12 12:00:57 2022] GET /identity/v3/role_assignments?group.id=8b050f7eb7bb48f4bc556655ec209b61&scope.project.id=4fe97b975d834d83a6733a0b2591bba4&role.id=8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:57.866722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cb1c3-2e32-4a4b-a0af-48f6e7ed763c None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:57.891363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:57.988137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6373ad-d74a-435f-898d-a472f6cf5e7f None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:58.004358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cb1c3-2e32-4a4b-a0af-48f6e7ed763c None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:58.035976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0150e5-7ee2-4aae-bb4a-9ff523b886f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:58.036621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0150e5-7ee2-4aae-bb4a-9ff523b886f9 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:58.036947 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0150e5-7ee2-4aae-bb4a-9ff523b886f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:58.037736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0150e5-7ee2-4aae-bb4a-9ff523b886f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:58.052166 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0150e5-7ee2-4aae-bb4a-9ff523b886f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:58.052527 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc549004-6350-463c-b939-49eb758e5b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:58.054289 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc549004-6350-463c-b939-49eb758e5b14 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:58.054758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc549004-6350-463c-b939-49eb758e5b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:58.055271 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc549004-6350-463c-b939-49eb758e5b14 None admin] PATH_INFO: `/v3/groups/8b050f7eb7bb48f4bc556655ec209b61` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:58.058604 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc549004-6350-463c-b939-49eb758e5b14 None admin] RBAC: Authorizing `identity:get_group(group_id=8b050f7eb7bb48f4bc556655ec209b61)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-6b0150e5-7ee2-4aae-bb4a-9ff523b886f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:58.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:58.059881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:58.060702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:58.060702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:58.060702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:58.060702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:58.060702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:58.060702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:58.060702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:58.060702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:58.060702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:58.060702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:58.060702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 12 12:00:58.060702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:00:58.060702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 179/359] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:58.061375 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cc7b1-52e1-449e-8d8c-aa45976f4e7b None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:58.061933 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cc7b1-52e1-449e-8d8c-aa45976f4e7b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:58.061933 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cc7b1-52e1-449e-8d8c-aa45976f4e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:58.063576 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc549004-6350-463c-b939-49eb758e5b14 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:58.065487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 181/360] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 12 12:00:57 2022] GET /identity/v3/groups/8b050f7eb7bb48f4bc556655ec209b61 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:58.066488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc549004-6350-463c-b939-49eb758e5b14 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:58.081168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-692cc7b1-52e1-449e-8d8c-aa45976f4e7b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:58.082390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-692cc7b1-52e1-449e-8d8c-aa45976f4e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:58.109010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:58.163980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 180/361] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:58.165502 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8994bb-40d8-4814-b9a4-a41aefffa176 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:58.165929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8994bb-40d8-4814-b9a4-a41aefffa176 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:58.166419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8994bb-40d8-4814-b9a4-a41aefffa176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:58.185972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-ba8994bb-40d8-4814-b9a4-a41aefffa176 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:00:58.187345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ba8994bb-40d8-4814-b9a4-a41aefffa176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:00:58.208067 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:58.208996 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc549004-6350-463c-b939-49eb758e5b14 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:58.250400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159d6f3-74b0-4c9b-a608-8855c7401b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:58.251493 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159d6f3-74b0-4c9b-a608-8855c7401b88 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:58.252014 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159d6f3-74b0-4c9b-a608-8855c7401b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:58.252474 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159d6f3-74b0-4c9b-a608-8855c7401b88 None admin] PATH_INFO: `/v3/groups/8b050f7eb7bb48f4bc556655ec209b61` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:58.253923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159d6f3-74b0-4c9b-a608-8855c7401b88 None admin] RBAC: Authorizing `identity:get_group(group_id=8b050f7eb7bb48f4bc556655ec209b61)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:58.255596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159d6f3-74b0-4c9b-a608-8855c7401b88 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:58.258763 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 182/362] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 12 12:00:58 2022] GET /identity/v3/groups/8b050f7eb7bb48f4bc556655ec209b61 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:58.259791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159d6f3-74b0-4c9b-a608-8855c7401b88 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:58.295006 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 181/363] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:00:58.296611 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8994bb-40d8-4814-b9a4-a41aefffa176 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:58.387514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:58.391705 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159d6f3-74b0-4c9b-a608-8855c7401b88 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:58.429817 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e6e00-1eca-40f4-9a8b-1502d7106520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:58.431059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e6e00-1eca-40f4-9a8b-1502d7106520 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:58.431540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e6e00-1eca-40f4-9a8b-1502d7106520 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:58.431996 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e6e00-1eca-40f4-9a8b-1502d7106520 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:58.432878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e6e00-1eca-40f4-9a8b-1502d7106520 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:58.433394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e6e00-1eca-40f4-9a8b-1502d7106520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:58.433494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8994bb-40d8-4814-b9a4-a41aefffa176 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:58.435173 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e6e00-1eca-40f4-9a8b-1502d7106520 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:58.446140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 183/364] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:00:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:58.447245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e6e00-1eca-40f4-9a8b-1502d7106520 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:58.469660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7799d3-9333-4dad-8f41-3363f5870d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:58.470319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7799d3-9333-4dad-8f41-3363f5870d19 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:58.470597 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7799d3-9333-4dad-8f41-3363f5870d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:58.470867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7799d3-9333-4dad-8f41-3363f5870d19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:58.471594 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7799d3-9333-4dad-8f41-3363f5870d19 None admin] RBAC: Adding query filter params (group.id=8b050f7eb7bb48f4bc556655ec209b61, role.id=b5e411fc96434df69c6cc1df951b3e6f, scope.project.id=4fe97b975d834d83a6733a0b2591bba4) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:58.471949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7799d3-9333-4dad-8f41-3363f5870d19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:58.473413 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7799d3-9333-4dad-8f41-3363f5870d19 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:58.480311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 182/365] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 12 12:00:58 2022] GET /identity/v3/role_assignments?group.id=8b050f7eb7bb48f4bc556655ec209b61&scope.project.id=4fe97b975d834d83a6733a0b2591bba4&role.id=b5e411fc96434df69c6cc1df951b3e6f => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:58.481084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7799d3-9333-4dad-8f41-3363f5870d19 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:58.510000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:58.573515 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e6e00-1eca-40f4-9a8b-1502d7106520 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:58.623748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4927f-8b47-46bf-aad5-3c0c110ffab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:58.624463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4927f-8b47-46bf-aad5-3c0c110ffab0 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:58.625259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4927f-8b47-46bf-aad5-3c0c110ffab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:58.625612 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4927f-8b47-46bf-aad5-3c0c110ffab0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:58.626738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4927f-8b47-46bf-aad5-3c0c110ffab0 None admin] RBAC: Adding query filter params (group.id=8b050f7eb7bb48f4bc556655ec209b61, role.id=8b2ee9b11a934b67aa3fd68c497fbdf7, scope.project.id=ad5d2677b95b428a93e499f4ced1b86b) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:58.627120 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4927f-8b47-46bf-aad5-3c0c110ffab0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:58.629161 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4927f-8b47-46bf-aad5-3c0c110ffab0 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:58.629495 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7799d3-9333-4dad-8f41-3363f5870d19 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:58.637244 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 184/366] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 12 12:00:58 2022] GET /identity/v3/role_assignments?group.id=8b050f7eb7bb48f4bc556655ec209b61&scope.project.id=ad5d2677b95b428a93e499f4ced1b86b&role.id=8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:58.638273 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4927f-8b47-46bf-aad5-3c0c110ffab0 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:58.671983 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043f00a-6e90-4988-8346-54c1de70a384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:58.673060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043f00a-6e90-4988-8346-54c1de70a384 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:58.673636 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043f00a-6e90-4988-8346-54c1de70a384 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:58.674126 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043f00a-6e90-4988-8346-54c1de70a384 None admin] PATH_INFO: `/v3/users/43115344775b4d1f80556897c437cc4e` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:58.675599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043f00a-6e90-4988-8346-54c1de70a384 None admin] RBAC: Authorizing `identity:get_user(user_id=43115344775b4d1f80556897c437cc4e)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:58.677426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043f00a-6e90-4988-8346-54c1de70a384 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:58.678319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:58.679982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 183/367] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:00:58 2022] GET /identity/v3/users/43115344775b4d1f80556897c437cc4e => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:58.686423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043f00a-6e90-4988-8346-54c1de70a384 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:58.767567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4927f-8b47-46bf-aad5-3c0c110ffab0 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:58.833033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:58.836802 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf4ea0-3660-4ee7-99fa-3e0855a184ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:58.837576 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf4ea0-3660-4ee7-99fa-3e0855a184ab None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:58.837859 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf4ea0-3660-4ee7-99fa-3e0855a184ab None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:58.838517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf4ea0-3660-4ee7-99fa-3e0855a184ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:58.849754 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043f00a-6e90-4988-8346-54c1de70a384 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:58.860315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf4ea0-3660-4ee7-99fa-3e0855a184ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-14bf4ea0-3660-4ee7-99fa-3e0855a184ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:58.866592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:58.867540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:58.868340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:58.868340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:58.868340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:58.868340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:58.868340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:58.868340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:58.868340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:58.868340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:58.868340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:58.868340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:58.868340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:00:58.868340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:58.868340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 185/368] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:00:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:58.869495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf4ea0-3660-4ee7-99fa-3e0855a184ab None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:58.908692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05722a00-65fd-4c23-b059-d06abcde5b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:58.908692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05722a00-65fd-4c23-b059-d06abcde5b3b None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:58.908692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05722a00-65fd-4c23-b059-d06abcde5b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:58.908692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05722a00-65fd-4c23-b059-d06abcde5b3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:58.909082 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05722a00-65fd-4c23-b059-d06abcde5b3b None admin] RBAC: Adding query filter params (role.id=bf2de8c80c5e4be79e1ea7adc3cbfc56, user.id=43115344775b4d1f80556897c437cc4e, scope.system=all) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:58.909082 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05722a00-65fd-4c23-b059-d06abcde5b3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:58.909082 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05722a00-65fd-4c23-b059-d06abcde5b3b None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:58.923771 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 184/369] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Aug 12 12:00:58 2022] GET /identity/v3/role_assignments?user.id=43115344775b4d1f80556897c437cc4e&scope.system=all&role.id=bf2de8c80c5e4be79e1ea7adc3cbfc56 => generated 510 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:58.924681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05722a00-65fd-4c23-b059-d06abcde5b3b None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:58.994622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:59.015598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf4ea0-3660-4ee7-99fa-3e0855a184ab None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:59.059593 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471095f-4798-42df-8cee-fa5391acc1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:59.061204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471095f-4798-42df-8cee-fa5391acc1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:59.061698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471095f-4798-42df-8cee-fa5391acc1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:59.062227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471095f-4798-42df-8cee-fa5391acc1d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:59.067091 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05722a00-65fd-4c23-b059-d06abcde5b3b None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:59.075436 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471095f-4798-42df-8cee-fa5391acc1d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-4471095f-4798-42df-8cee-fa5391acc1d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:00:59.081638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:00:59.082581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:00:59.083358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:00:59.083358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:00:59.083358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:00:59.083358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:00:59.083358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:00:59.083358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:00:59.083358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:00:59.083358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:00:59.083358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:00:59.083358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:00:59.083358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:00:59.083358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:00:59.086391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 186/370] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:00:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:00:59.093879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471095f-4798-42df-8cee-fa5391acc1d4 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:59.108696 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047c740-f423-4a26-b50e-3cc76c01b934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:59.108696 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047c740-f423-4a26-b50e-3cc76c01b934 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:59.108696 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047c740-f423-4a26-b50e-3cc76c01b934 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:59.108696 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047c740-f423-4a26-b50e-3cc76c01b934 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:59.109419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047c740-f423-4a26-b50e-3cc76c01b934 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:59.109419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047c740-f423-4a26-b50e-3cc76c01b934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:59.110277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047c740-f423-4a26-b50e-3cc76c01b934 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:59.121065 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 185/371] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:00:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:59.127085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047c740-f423-4a26-b50e-3cc76c01b934 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:59.138480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:59.249227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047c740-f423-4a26-b50e-3cc76c01b934 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:59.287293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d207eb-87f1-4e58-a8f6-965da7534458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:59.287919 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d207eb-87f1-4e58-a8f6-965da7534458 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:59.288204 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d207eb-87f1-4e58-a8f6-965da7534458 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:59.288470 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d207eb-87f1-4e58-a8f6-965da7534458 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:59.293436 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471095f-4798-42df-8cee-fa5391acc1d4 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:59.367169 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d207eb-87f1-4e58-a8f6-965da7534458 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:59.369147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d207eb-87f1-4e58-a8f6-965da7534458 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:59.405415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:59.407827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372433c-ab16-42c1-9a98-b003e8b18cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:59.408662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372433c-ab16-42c1-9a98-b003e8b18cbb None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:59.408880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372433c-ab16-42c1-9a98-b003e8b18cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:59.409155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372433c-ab16-42c1-9a98-b003e8b18cbb None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:59.409831 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372433c-ab16-42c1-9a98-b003e8b18cbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:59.410136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372433c-ab16-42c1-9a98-b003e8b18cbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:59.411585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372433c-ab16-42c1-9a98-b003e8b18cbb None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:59.428755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 187/372] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:00:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:00:59.435552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372433c-ab16-42c1-9a98-b003e8b18cbb None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:59.453398 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:59.484032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 186/373] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 12 12:00:59.489927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d207eb-87f1-4e58-a8f6-965da7534458 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:59.603910 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d207eb-87f1-4e58-a8f6-965da7534458 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:59.612409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:59.643140 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8be2a-9893-4f6d-92f3-c0fd88bd8948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:59.643866 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8be2a-9893-4f6d-92f3-c0fd88bd8948 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:59.644304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8be2a-9893-4f6d-92f3-c0fd88bd8948 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:59.644583 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8be2a-9893-4f6d-92f3-c0fd88bd8948 None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:59.646275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8be2a-9893-4f6d-92f3-c0fd88bd8948 None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:59.647918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8be2a-9893-4f6d-92f3-c0fd88bd8948 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:59.650161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 187/374] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:00:59 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:59.656178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8be2a-9893-4f6d-92f3-c0fd88bd8948 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:59.666926 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372433c-ab16-42c1-9a98-b003e8b18cbb None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:59.749840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02b664-944f-43da-96da-996f64c64cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:59.750452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02b664-944f-43da-96da-996f64c64cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:59.750732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02b664-944f-43da-96da-996f64c64cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:59.751014 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02b664-944f-43da-96da-996f64c64cc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:59.766075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8be2a-9893-4f6d-92f3-c0fd88bd8948 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:59.806570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3905ac3-c43b-4ea6-a231-886500114e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:59.807204 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3905ac3-c43b-4ea6-a231-886500114e0d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:59.807482 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3905ac3-c43b-4ea6-a231-886500114e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:59.807748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3905ac3-c43b-4ea6-a231-886500114e0d None admin] PATH_INFO: `/v3/groups/8b050f7eb7bb48f4bc556655ec209b61` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:59.808931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3905ac3-c43b-4ea6-a231-886500114e0d None admin] RBAC: Authorizing `identity:get_group(group_id=8b050f7eb7bb48f4bc556655ec209b61)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:59.811794 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3905ac3-c43b-4ea6-a231-886500114e0d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:59.813892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 188/375] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 12 12:00:59 2022] GET /identity/v3/groups/8b050f7eb7bb48f4bc556655ec209b61 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:59.819484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3905ac3-c43b-4ea6-a231-886500114e0d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:00:59.874481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:00:59.934914 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3905ac3-c43b-4ea6-a231-886500114e0d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:00:59.965476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02b664-944f-43da-96da-996f64c64cc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:59.966959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02b664-944f-43da-96da-996f64c64cc8 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:59.973201 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa651b-114b-41b1-bdb0-96276f27360c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:00:59.973813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa651b-114b-41b1-bdb0-96276f27360c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:00:59.974089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa651b-114b-41b1-bdb0-96276f27360c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:00:59.974411 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa651b-114b-41b1-bdb0-96276f27360c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:00:59.975869 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa651b-114b-41b1-bdb0-96276f27360c None admin] RBAC: Adding query filter params (group.id=8b050f7eb7bb48f4bc556655ec209b61, role.id=b5e411fc96434df69c6cc1df951b3e6f, scope.project.id=ad5d2677b95b428a93e499f4ced1b86b) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:00:59.976178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa651b-114b-41b1-bdb0-96276f27360c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:00:59.978120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa651b-114b-41b1-bdb0-96276f27360c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:00:59.984557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 189/376] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 12 12:00:59 2022] GET /identity/v3/role_assignments?group.id=8b050f7eb7bb48f4bc556655ec209b61&scope.project.id=ad5d2677b95b428a93e499f4ced1b86b&role.id=b5e411fc96434df69c6cc1df951b3e6f => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:00:59.984971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:00.109561 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 188/377] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:00:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 12 12:01:00.116684 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa651b-114b-41b1-bdb0-96276f27360c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:00.226369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa651b-114b-41b1-bdb0-96276f27360c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:00.234034 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:00.262813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ed994-debd-461b-ab8e-cfab5fd16f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:00.263444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ed994-debd-461b-ab8e-cfab5fd16f2f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:00.263729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ed994-debd-461b-ab8e-cfab5fd16f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:00.264000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ed994-debd-461b-ab8e-cfab5fd16f2f None admin] PATH_INFO: `/v3/projects/9e93c7bb262f46249ca80b95c794bf2f` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:00.265146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ed994-debd-461b-ab8e-cfab5fd16f2f None admin] RBAC: Authorizing `identity:get_project(project_id=9e93c7bb262f46249ca80b95c794bf2f)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:00.266596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ed994-debd-461b-ab8e-cfab5fd16f2f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:00.268598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 190/378] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:01:00 2022] GET /identity/v3/projects/9e93c7bb262f46249ca80b95c794bf2f => generated 344 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:00.275355 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02b664-944f-43da-96da-996f64c64cc8 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:00.378963 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02b664-944f-43da-96da-996f64c64cc8 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:00.389253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:00.415017 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18346728-e5c3-4b41-964a-3fe3679a5ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:00.415758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18346728-e5c3-4b41-964a-3fe3679a5ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:00.416004 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18346728-e5c3-4b41-964a-3fe3679a5ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:00.416269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18346728-e5c3-4b41-964a-3fe3679a5ad3 None admin] PATH_INFO: `/v3/groups/23bd955784184c7488dd4fed6e261c0a` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:00.422514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18346728-e5c3-4b41-964a-3fe3679a5ad3 None admin] RBAC: Authorizing `identity:get_group(group_id=23bd955784184c7488dd4fed6e261c0a)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:00.424086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18346728-e5c3-4b41-964a-3fe3679a5ad3 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:00.426141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 189/379] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 12 12:01:00 2022] GET /identity/v3/groups/23bd955784184c7488dd4fed6e261c0a => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:00.432921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ed994-debd-461b-ab8e-cfab5fd16f2f None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:00.509161 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d9fc2-f6cf-4c25-ac43-6d8ee763a852 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:00.509448 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d9fc2-f6cf-4c25-ac43-6d8ee763a852 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:00.509690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d9fc2-f6cf-4c25-ac43-6d8ee763a852 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:00.511281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 190/380] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:00.525105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f2503-a0fb-4e60-b4f3-936249723ccf None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:00.525578 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f2503-a0fb-4e60-b4f3-936249723ccf None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:00.525952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f2503-a0fb-4e60-b4f3-936249723ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:00.550285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-663f2503-a0fb-4e60-b4f3-936249723ccf None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:00.551992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-663f2503-a0fb-4e60-b4f3-936249723ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:00.558036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ed994-debd-461b-ab8e-cfab5fd16f2f None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:00.620187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210eaa1c-6814-4d7f-a8b0-bb75a026cc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:00.621073 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210eaa1c-6814-4d7f-a8b0-bb75a026cc41 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:00.621526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210eaa1c-6814-4d7f-a8b0-bb75a026cc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:00.621945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210eaa1c-6814-4d7f-a8b0-bb75a026cc41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:00.622923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210eaa1c-6814-4d7f-a8b0-bb75a026cc41 None admin] RBAC: Adding query filter params (group.id=23bd955784184c7488dd4fed6e261c0a, role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=9e93c7bb262f46249ca80b95c794bf2f) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:00.623366 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210eaa1c-6814-4d7f-a8b0-bb75a026cc41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:00.624975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210eaa1c-6814-4d7f-a8b0-bb75a026cc41 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:00.638169 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 191/381] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 12 12:01:00 2022] GET /identity/v3/role_assignments?group.id=23bd955784184c7488dd4fed6e261c0a&scope.project.id=9e93c7bb262f46249ca80b95c794bf2f&role.id=906e3a8a6c7540a38f629f2567133347 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:00.638765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 191/382] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:01.115538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210eaa1c-6814-4d7f-a8b0-bb75a026cc41 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:01.167772 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:01.221020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71caa8ed-e32a-4b95-ae24-771ea1c7fa90 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:01.221553 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71caa8ed-e32a-4b95-ae24-771ea1c7fa90 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:01.221553 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71caa8ed-e32a-4b95-ae24-771ea1c7fa90 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:01.223061 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 192/383] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:01.229045 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974aea53-e467-4afc-ad8c-f06d08a870a9 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:01.229325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974aea53-e467-4afc-ad8c-f06d08a870a9 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:01.229600 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974aea53-e467-4afc-ad8c-f06d08a870a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:01.256450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210eaa1c-6814-4d7f-a8b0-bb75a026cc41 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:01.261977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-974aea53-e467-4afc-ad8c-f06d08a870a9 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:01.263165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-974aea53-e467-4afc-ad8c-f06d08a870a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:01.324016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab9868-67c0-45f8-a6ca-adbf49d3a9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:01.324788 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab9868-67c0-45f8-a6ca-adbf49d3a9ae None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:01.325178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab9868-67c0-45f8-a6ca-adbf49d3a9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:01.325458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab9868-67c0-45f8-a6ca-adbf49d3a9ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:01.342465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab9868-67c0-45f8-a6ca-adbf49d3a9ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-6eab9868-67c0-45f8-a6ca-adbf49d3a9ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:01.347878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:01.348844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:01.350187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:01.350187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:01.350187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:01.350187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:01.350187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:01.350187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:01.350187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:01.350187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:01.350187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:01.350187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:01.350187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:01:01.350187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:01.353750 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 192/384] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:01.360252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab9868-67c0-45f8-a6ca-adbf49d3a9ae None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:01.360894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 193/385] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:01:01.494323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:01.497362 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0fd5a-2752-48b2-9c11-5f142bfa4062 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:01.497755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0fd5a-2752-48b2-9c11-5f142bfa4062 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:01.498127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0fd5a-2752-48b2-9c11-5f142bfa4062 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:01.499274 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab9868-67c0-45f8-a6ca-adbf49d3a9ae None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:01.499951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 194/386] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:01.505007 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b688763-26f8-4550-923b-d1d4db4da572 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:01.511506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b688763-26f8-4550-923b-d1d4db4da572 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:01.511506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b688763-26f8-4550-923b-d1d4db4da572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:01.528991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-0b688763-26f8-4550-923b-d1d4db4da572 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:01.530394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0b688763-26f8-4550-923b-d1d4db4da572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:01.544462 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f1869-292f-4a0e-ad84-247025c5fd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:01.545416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f1869-292f-4a0e-ad84-247025c5fd9c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:01.545932 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f1869-292f-4a0e-ad84-247025c5fd9c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:01.546338 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f1869-292f-4a0e-ad84-247025c5fd9c None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:01.547142 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f1869-292f-4a0e-ad84-247025c5fd9c None admin] RBAC: Adding query filter params (name=member) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:01.547575 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f1869-292f-4a0e-ad84-247025c5fd9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:01.549380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f1869-292f-4a0e-ad84-247025c5fd9c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:01.558967 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:01.561081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 193/387] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:01:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:01.566942 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f1869-292f-4a0e-ad84-247025c5fd9c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:01.608099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 195/388] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:01.678506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f1869-292f-4a0e-ad84-247025c5fd9c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:01.718455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b7c98-4060-4b1f-9b8c-0c3cd7e293db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:01.719347 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b7c98-4060-4b1f-9b8c-0c3cd7e293db None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:01.719813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b7c98-4060-4b1f-9b8c-0c3cd7e293db None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:01.720230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b7c98-4060-4b1f-9b8c-0c3cd7e293db None admin] PATH_INFO: `/v3/groups/8b050f7eb7bb48f4bc556655ec209b61` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:01.722914 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b7c98-4060-4b1f-9b8c-0c3cd7e293db None admin] RBAC: Authorizing `identity:get_group(group_id=8b050f7eb7bb48f4bc556655ec209b61)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:01.724633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b7c98-4060-4b1f-9b8c-0c3cd7e293db None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:01.727432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 194/389] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 12 12:01:01 2022] GET /identity/v3/groups/8b050f7eb7bb48f4bc556655ec209b61 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:01.733176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b688763-26f8-4550-923b-d1d4db4da572 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:01.868330 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b688763-26f8-4550-923b-d1d4db4da572 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:01.905457 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b7c98-4060-4b1f-9b8c-0c3cd7e293db None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:01.909678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bc084-52c2-48c0-93cc-d49c9db3dcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:01.910545 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bc084-52c2-48c0-93cc-d49c9db3dcfa None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:01.911020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bc084-52c2-48c0-93cc-d49c9db3dcfa None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:01.911505 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bc084-52c2-48c0-93cc-d49c9db3dcfa None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:01.914302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bc084-52c2-48c0-93cc-d49c9db3dcfa None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:01.915978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bc084-52c2-48c0-93cc-d49c9db3dcfa None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:01.918307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 196/390] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:01:01 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:01.935336 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bc084-52c2-48c0-93cc-d49c9db3dcfa None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:01.965025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:01.980994 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:02.051532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b7c98-4060-4b1f-9b8c-0c3cd7e293db None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:02.055051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bc084-52c2-48c0-93cc-d49c9db3dcfa None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:02.090230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4406810-4080-4681-9f74-48fc775ef98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:02.090782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4406810-4080-4681-9f74-48fc775ef98b None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:02.091063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4406810-4080-4681-9f74-48fc775ef98b None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:02.091172 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752d9dd-b741-4c8f-9ba2-da73a0c5881d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:02.091336 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4406810-4080-4681-9f74-48fc775ef98b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:02.091988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752d9dd-b741-4c8f-9ba2-da73a0c5881d None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:02.092364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752d9dd-b741-4c8f-9ba2-da73a0c5881d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:02.092757 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752d9dd-b741-4c8f-9ba2-da73a0c5881d None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4/groups/8b050f7eb7bb48f4bc556655ec209b61/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:02.094760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752d9dd-b741-4c8f-9ba2-da73a0c5881d None admin] RBAC: Authorizing `identity:create_grant(project_id=4fe97b975d834d83a6733a0b2591bba4, group_id=8b050f7eb7bb48f4bc556655ec209b61, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:02.096292 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752d9dd-b741-4c8f-9ba2-da73a0c5881d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:02.103506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4406810-4080-4681-9f74-48fc775ef98b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-e4406810-4080-4681-9f74-48fc775ef98b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:02.109768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:02.113053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:02.113815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:02.113815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:02.113815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:02.113815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:02.113815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:02.113815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:02.113815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:02.113815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:02.113815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:02.113815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:02.113815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:01:02.113815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:02.113815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 195/391] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:01:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:02.113815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4406810-4080-4681-9f74-48fc775ef98b None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:02.170708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 197/392] 10.222.0.27 () {58 vars in 1442 bytes} [Fri Aug 12 12:01:01 2022] PUT /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4/groups/8b050f7eb7bb48f4bc556655ec209b61/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:01:02.171870 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752d9dd-b741-4c8f-9ba2-da73a0c5881d None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:02.237655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:02.253156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4406810-4080-4681-9f74-48fc775ef98b None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:02.292968 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e86ca2-a800-497e-90d4-28e383d29932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:02.293947 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e86ca2-a800-497e-90d4-28e383d29932 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:02.294418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e86ca2-a800-497e-90d4-28e383d29932 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:02.295037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e86ca2-a800-497e-90d4-28e383d29932 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:02.308044 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e86ca2-a800-497e-90d4-28e383d29932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-49e86ca2-a800-497e-90d4-28e383d29932 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:02.314002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:02.314888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:02.315655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:02.315655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:02.315655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:02.315655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:02.315655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:02.315655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:02.315655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:02.315655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:02.315655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:02.315655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:02.315655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:01:02.315655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:02.318901 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 196/393] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:02.325199 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e86ca2-a800-497e-90d4-28e383d29932 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:02.405699 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:02.416258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752d9dd-b741-4c8f-9ba2-da73a0c5881d None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:02.439507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e86ca2-a800-497e-90d4-28e383d29932 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:02.464153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd82f81-be8f-4e5e-998f-4f423be75974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:02.466377 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd82f81-be8f-4e5e-998f-4f423be75974 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:02.466377 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd82f81-be8f-4e5e-998f-4f423be75974 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:02.466377 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd82f81-be8f-4e5e-998f-4f423be75974 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:02.466377 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd82f81-be8f-4e5e-998f-4f423be75974 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:02.466377 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd82f81-be8f-4e5e-998f-4f423be75974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:02.468128 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd82f81-be8f-4e5e-998f-4f423be75974 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:02.477986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 198/394] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:02.479442 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067b2b8-eba3-4cee-b6dd-1213eafc74e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:02.480265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067b2b8-eba3-4cee-b6dd-1213eafc74e1 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:02.480796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067b2b8-eba3-4cee-b6dd-1213eafc74e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:02.481297 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067b2b8-eba3-4cee-b6dd-1213eafc74e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:02.482396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067b2b8-eba3-4cee-b6dd-1213eafc74e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:02.482895 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067b2b8-eba3-4cee-b6dd-1213eafc74e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:02.484486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067b2b8-eba3-4cee-b6dd-1213eafc74e1 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:02.485633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd82f81-be8f-4e5e-998f-4f423be75974 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:02.495016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 197/395] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:01:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:02.500794 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067b2b8-eba3-4cee-b6dd-1213eafc74e1 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:02.514155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:02.620484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067b2b8-eba3-4cee-b6dd-1213eafc74e1 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:02.654432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd82f81-be8f-4e5e-998f-4f423be75974 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:02.699214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc07d1-e02e-4ad8-9074-bd26e9ac532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:02.699214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc07d1-e02e-4ad8-9074-bd26e9ac532c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:02.699214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc07d1-e02e-4ad8-9074-bd26e9ac532c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:02.699214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc07d1-e02e-4ad8-9074-bd26e9ac532c None admin] PATH_INFO: `/v3/groups/8b050f7eb7bb48f4bc556655ec209b61` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:02.699701 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc07d1-e02e-4ad8-9074-bd26e9ac532c None admin] RBAC: Authorizing `identity:get_group(group_id=8b050f7eb7bb48f4bc556655ec209b61)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:02.701877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc07d1-e02e-4ad8-9074-bd26e9ac532c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:02.705205 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51d7c3-7076-4951-a021-b0728a6733d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:02.706005 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51d7c3-7076-4951-a021-b0728a6733d6 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:02.706443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 199/396] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 12 12:01:02 2022] GET /identity/v3/groups/8b050f7eb7bb48f4bc556655ec209b61 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:02.706865 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51d7c3-7076-4951-a021-b0728a6733d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:02.707289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51d7c3-7076-4951-a021-b0728a6733d6 None admin] PATH_INFO: `/v3/groups/8b050f7eb7bb48f4bc556655ec209b61` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:02.710358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc07d1-e02e-4ad8-9074-bd26e9ac532c None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:02.710743 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51d7c3-7076-4951-a021-b0728a6733d6 None admin] RBAC: Authorizing `identity:get_group(group_id=8b050f7eb7bb48f4bc556655ec209b61)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:02.712735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51d7c3-7076-4951-a021-b0728a6733d6 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:02.717986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 198/397] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 12 12:01:02 2022] GET /identity/v3/groups/8b050f7eb7bb48f4bc556655ec209b61 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:02.722465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51d7c3-7076-4951-a021-b0728a6733d6 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:02.753587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:02.847909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51d7c3-7076-4951-a021-b0728a6733d6 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:02.869042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:02.890079 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18934004-86f9-4b17-8105-fa505ed5c050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:02.890722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18934004-86f9-4b17-8105-fa505ed5c050 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:02.891096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18934004-86f9-4b17-8105-fa505ed5c050 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:02.891526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18934004-86f9-4b17-8105-fa505ed5c050 None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:02.892920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18934004-86f9-4b17-8105-fa505ed5c050 None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:02.894445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18934004-86f9-4b17-8105-fa505ed5c050 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:02.896388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 199/398] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:01:02 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:02.907217 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18934004-86f9-4b17-8105-fa505ed5c050 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:02.940304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc07d1-e02e-4ad8-9074-bd26e9ac532c None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:03.012778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bac301-d5f1-4634-9220-85bf8db84aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:03.013476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bac301-d5f1-4634-9220-85bf8db84aaa None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:03.013822 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bac301-d5f1-4634-9220-85bf8db84aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:03.014163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bac301-d5f1-4634-9220-85bf8db84aaa None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:03.016442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bac301-d5f1-4634-9220-85bf8db84aaa None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:03.017974 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bac301-d5f1-4634-9220-85bf8db84aaa None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:03.019707 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 200/399] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:01:02 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:03.021096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3528214-e7c8-4311-9cce-8eb3a028e211 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:03.021493 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3528214-e7c8-4311-9cce-8eb3a028e211 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:03.021839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3528214-e7c8-4311-9cce-8eb3a028e211 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:03.023383 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 201/400] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:03.030324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3528214-e7c8-4311-9cce-8eb3a028e211 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:03.032193 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18934004-86f9-4b17-8105-fa505ed5c050 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:03.074961 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78897e-5108-4e78-869e-ef25a63c3e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:03.075812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78897e-5108-4e78-869e-ef25a63c3e28 None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:03.076097 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78897e-5108-4e78-869e-ef25a63c3e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:03.076433 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78897e-5108-4e78-869e-ef25a63c3e28 None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b/groups/8b050f7eb7bb48f4bc556655ec209b61/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:03.078576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78897e-5108-4e78-869e-ef25a63c3e28 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad5d2677b95b428a93e499f4ced1b86b, group_id=8b050f7eb7bb48f4bc556655ec209b61, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:03.080132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78897e-5108-4e78-869e-ef25a63c3e28 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:03.145205 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:03.158949 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3528214-e7c8-4311-9cce-8eb3a028e211 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:03.169346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 200/401] 10.222.0.27 () {58 vars in 1442 bytes} [Fri Aug 12 12:01:02 2022] PUT /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b/groups/8b050f7eb7bb48f4bc556655ec209b61/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:01:03.170311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3196378-61e6-4664-94a4-6496343c5661 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:03.170584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3196378-61e6-4664-94a4-6496343c5661 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:03.173961 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3196378-61e6-4664-94a4-6496343c5661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:03.203976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:03.214275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-b3196378-61e6-4664-94a4-6496343c5661 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:03.217147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b3196378-61e6-4664-94a4-6496343c5661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:03.231392 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595fbf3-b6da-4bd1-adba-4f390d650e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:03.232993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595fbf3-b6da-4bd1-adba-4f390d650e75 None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:03.232993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595fbf3-b6da-4bd1-adba-4f390d650e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:03.232993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595fbf3-b6da-4bd1-adba-4f390d650e75 None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4/groups/8b050f7eb7bb48f4bc556655ec209b61/roles/b5e411fc96434df69c6cc1df951b3e6f` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:03.235352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595fbf3-b6da-4bd1-adba-4f390d650e75 None admin] RBAC: Authorizing `identity:create_grant(project_id=4fe97b975d834d83a6733a0b2591bba4, group_id=8b050f7eb7bb48f4bc556655ec209b61, role_id=b5e411fc96434df69c6cc1df951b3e6f)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:03.236990 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595fbf3-b6da-4bd1-adba-4f390d650e75 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:03.294757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 201/402] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:03.296583 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523389e-c757-428f-86e4-344f2693d22e None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:03.296921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523389e-c757-428f-86e4-344f2693d22e None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:03.297166 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523389e-c757-428f-86e4-344f2693d22e None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:03.298583 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 202/403] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:03.303842 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04105e65-0ad5-4e1d-b3a7-88cbb14fdf7c None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:03.304130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04105e65-0ad5-4e1d-b3a7-88cbb14fdf7c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:03.304396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04105e65-0ad5-4e1d-b3a7-88cbb14fdf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:03.324356 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-04105e65-0ad5-4e1d-b3a7-88cbb14fdf7c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:03.325654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-04105e65-0ad5-4e1d-b3a7-88cbb14fdf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:03.377592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 202/404] 10.222.0.27 () {58 vars in 1442 bytes} [Fri Aug 12 12:01:03 2022] PUT /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4/groups/8b050f7eb7bb48f4bc556655ec209b61/roles/b5e411fc96434df69c6cc1df951b3e6f => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:01:03.405489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 203/405] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:01:03.642824 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595fbf3-b6da-4bd1-adba-4f390d650e75 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:03.702539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04105e65-0ad5-4e1d-b3a7-88cbb14fdf7c None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:03.810908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595fbf3-b6da-4bd1-adba-4f390d650e75 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:03.833137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:03.847682 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb08ecd-9d59-4af8-b725-571c6411cdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:03.856323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb08ecd-9d59-4af8-b725-571c6411cdfa None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:03.856323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb08ecd-9d59-4af8-b725-571c6411cdfa None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:03.856323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb08ecd-9d59-4af8-b725-571c6411cdfa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:03.862067 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb08ecd-9d59-4af8-b725-571c6411cdfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-0bb08ecd-9d59-4af8-b725-571c6411cdfa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:03.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:03.869082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:03.869964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:03.869964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:03.869964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:03.869964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:03.869964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:03.869964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:03.869964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:03.869964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:03.869964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:03.869964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:03.869964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:01:03.869964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:03.874416 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 203/406] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:01:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:03.877001 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb08ecd-9d59-4af8-b725-571c6411cdfa None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:03.889929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:03.911371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04105e65-0ad5-4e1d-b3a7-88cbb14fdf7c None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:03.954490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d06be-9378-4325-b52e-5e5e9f47892d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:03.954490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d06be-9378-4325-b52e-5e5e9f47892d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:03.954490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d06be-9378-4325-b52e-5e5e9f47892d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:03.954490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d06be-9378-4325-b52e-5e5e9f47892d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:03.977284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d06be-9378-4325-b52e-5e5e9f47892d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-f59d06be-9378-4325-b52e-5e5e9f47892d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:03.982021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:03.982966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:03.983714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:03.983714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:03.983714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:03.983714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:03.983714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:03.983714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:03.983714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:03.983714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:03.983714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:03.983714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:03.983714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:03.983714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:03.983714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 204/407] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:01:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:03.990412 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d06be-9378-4325-b52e-5e5e9f47892d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:04.073940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb08ecd-9d59-4af8-b725-571c6411cdfa None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:04.140549 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08377cbe-796f-4fba-a210-be8a8c06fd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:04.141188 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08377cbe-796f-4fba-a210-be8a8c06fd86 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:04.141472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08377cbe-796f-4fba-a210-be8a8c06fd86 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:04.141748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08377cbe-796f-4fba-a210-be8a8c06fd86 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:04.142421 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08377cbe-796f-4fba-a210-be8a8c06fd86 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:04.142727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08377cbe-796f-4fba-a210-be8a8c06fd86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:04.144099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08377cbe-796f-4fba-a210-be8a8c06fd86 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:04.149685 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d06be-9378-4325-b52e-5e5e9f47892d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:04.169229 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 204/408] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:04.177288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08377cbe-796f-4fba-a210-be8a8c06fd86 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:04.197044 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:04.215700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b728bee-2e14-4e39-9b9b-e393a89ab506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:04.216326 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b728bee-2e14-4e39-9b9b-e393a89ab506 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:04.216612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b728bee-2e14-4e39-9b9b-e393a89ab506 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:04.216912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b728bee-2e14-4e39-9b9b-e393a89ab506 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:04.217642 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b728bee-2e14-4e39-9b9b-e393a89ab506 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:04.217956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b728bee-2e14-4e39-9b9b-e393a89ab506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:04.219525 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b728bee-2e14-4e39-9b9b-e393a89ab506 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:04.235994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 205/409] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:04.242557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b728bee-2e14-4e39-9b9b-e393a89ab506 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:04.289476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:04.353594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08377cbe-796f-4fba-a210-be8a8c06fd86 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:04.394136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8eff37-4864-4241-8202-c258c184ee2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:04.395201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8eff37-4864-4241-8202-c258c184ee2d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:04.395683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8eff37-4864-4241-8202-c258c184ee2d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:04.397462 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8eff37-4864-4241-8202-c258c184ee2d None admin] PATH_INFO: `/v3/groups/8b050f7eb7bb48f4bc556655ec209b61` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:04.398797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8eff37-4864-4241-8202-c258c184ee2d None admin] RBAC: Authorizing `identity:get_group(group_id=8b050f7eb7bb48f4bc556655ec209b61)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:04.401309 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8eff37-4864-4241-8202-c258c184ee2d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:04.403678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 205/410] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 12 12:01:04 2022] GET /identity/v3/groups/8b050f7eb7bb48f4bc556655ec209b61 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:04.409350 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bbc78-43dc-400e-b4a0-c36d0b6bd070 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:04.409861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bbc78-43dc-400e-b4a0-c36d0b6bd070 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:04.410340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7bbc78-43dc-400e-b4a0-c36d0b6bd070 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:04.412037 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 206/411] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:04.413016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bbc78-43dc-400e-b4a0-c36d0b6bd070 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:04.427889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b728bee-2e14-4e39-9b9b-e393a89ab506 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:04.468630 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d957c-7164-4571-aa25-95065b28ae22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:04.469280 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d957c-7164-4571-aa25-95065b28ae22 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:04.469561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d957c-7164-4571-aa25-95065b28ae22 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:04.469834 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d957c-7164-4571-aa25-95065b28ae22 None admin] PATH_INFO: `/v3/groups/23bd955784184c7488dd4fed6e261c0a` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:04.472065 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d957c-7164-4571-aa25-95065b28ae22 None admin] RBAC: Authorizing `identity:get_group(group_id=23bd955784184c7488dd4fed6e261c0a)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:04.473557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d957c-7164-4571-aa25-95065b28ae22 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:04.475473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 206/412] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 12 12:01:04 2022] GET /identity/v3/groups/23bd955784184c7488dd4fed6e261c0a => generated 229 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:04.480560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc815eed-53d0-4876-97cc-0b29daba06a5 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:04.480560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc815eed-53d0-4876-97cc-0b29daba06a5 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:04.480560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc815eed-53d0-4876-97cc-0b29daba06a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:04.496963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-bc815eed-53d0-4876-97cc-0b29daba06a5 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:04.498282 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-bc815eed-53d0-4876-97cc-0b29daba06a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:04.551455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:04.553691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7bbc78-43dc-400e-b4a0-c36d0b6bd070 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:04.573130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 207/413] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:04.574377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc815eed-53d0-4876-97cc-0b29daba06a5 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:04.601577 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b8a87-937b-44cf-94b2-96df66a69d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:04.602354 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b8a87-937b-44cf-94b2-96df66a69d9a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:04.602654 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b8a87-937b-44cf-94b2-96df66a69d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:04.602933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b8a87-937b-44cf-94b2-96df66a69d9a None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:04.604456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b8a87-937b-44cf-94b2-96df66a69d9a None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:04.606365 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b8a87-937b-44cf-94b2-96df66a69d9a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:04.608610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 207/414] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:01:04 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:04.617510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b8a87-937b-44cf-94b2-96df66a69d9a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:04.625139 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:04.749856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b8a87-937b-44cf-94b2-96df66a69d9a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:04.793729 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5b7e2-7389-4855-a157-4019a201c20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:04.794643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5b7e2-7389-4855-a157-4019a201c20d None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:04.794931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5b7e2-7389-4855-a157-4019a201c20d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:04.795207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5b7e2-7389-4855-a157-4019a201c20d None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b/groups/8b050f7eb7bb48f4bc556655ec209b61/roles/b5e411fc96434df69c6cc1df951b3e6f` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:04.798788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5b7e2-7389-4855-a157-4019a201c20d None admin] RBAC: Authorizing `identity:create_grant(project_id=ad5d2677b95b428a93e499f4ced1b86b, group_id=8b050f7eb7bb48f4bc556655ec209b61, role_id=b5e411fc96434df69c6cc1df951b3e6f)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:04.800248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5b7e2-7389-4855-a157-4019a201c20d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:04.804625 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc815eed-53d0-4876-97cc-0b29daba06a5 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:04.846146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbb615-e243-4393-93b4-b839e7321251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:04.846779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbb615-e243-4393-93b4-b839e7321251 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:04.847053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbb615-e243-4393-93b4-b839e7321251 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:04.847386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbb615-e243-4393-93b4-b839e7321251 None admin] PATH_INFO: `/v3/projects/9e93c7bb262f46249ca80b95c794bf2f` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:04.848505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbb615-e243-4393-93b4-b839e7321251 None admin] RBAC: Authorizing `identity:get_project(project_id=9e93c7bb262f46249ca80b95c794bf2f)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:04.850189 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbb615-e243-4393-93b4-b839e7321251 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:04.852292 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 208/415] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:01:04 2022] GET /identity/v3/projects/9e93c7bb262f46249ca80b95c794bf2f => generated 344 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:04.858145 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbb615-e243-4393-93b4-b839e7321251 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:04.873297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 208/416] 10.222.0.27 () {58 vars in 1442 bytes} [Fri Aug 12 12:01:04 2022] PUT /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b/groups/8b050f7eb7bb48f4bc556655ec209b61/roles/b5e411fc96434df69c6cc1df951b3e6f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:01:04.928571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5b7e2-7389-4855-a157-4019a201c20d None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:04.934674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:05.006816 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbb615-e243-4393-93b4-b839e7321251 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:05.053066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27a633-762d-47cc-807a-c70afb7e0c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:05.055182 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27a633-762d-47cc-807a-c70afb7e0c83 None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:05.056169 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27a633-762d-47cc-807a-c70afb7e0c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:05.056610 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27a633-762d-47cc-807a-c70afb7e0c83 None admin] PATH_INFO: `/v3/projects/9e93c7bb262f46249ca80b95c794bf2f/groups/23bd955784184c7488dd4fed6e261c0a/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:05.062130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27a633-762d-47cc-807a-c70afb7e0c83 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e93c7bb262f46249ca80b95c794bf2f, group_id=23bd955784184c7488dd4fed6e261c0a, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:05.064360 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27a633-762d-47cc-807a-c70afb7e0c83 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:05.085637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:05.125597 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5b7e2-7389-4855-a157-4019a201c20d None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:05.146659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 209/417] 10.222.0.27 () {58 vars in 1442 bytes} [Fri Aug 12 12:01:04 2022] PUT /identity/v3/projects/9e93c7bb262f46249ca80b95c794bf2f/groups/23bd955784184c7488dd4fed6e261c0a/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:01:05.170193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc139dc-e165-4fc1-bf93-46630eca7c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:05.176066 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc139dc-e165-4fc1-bf93-46630eca7c14 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:05.176066 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc139dc-e165-4fc1-bf93-46630eca7c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:05.176066 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc139dc-e165-4fc1-bf93-46630eca7c14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:05.184933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc139dc-e165-4fc1-bf93-46630eca7c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-edc139dc-e165-4fc1-bf93-46630eca7c14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:05.190687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:05.191609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:05.192381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:05.192381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:05.192381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:05.192381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:05.192381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:05.192381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:05.192381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:05.192381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:05.192381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:05.192381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:05.192381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:01:05.192381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:05.192381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 209/418] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:05.200426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27a633-762d-47cc-807a-c70afb7e0c83 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:05.366307 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27a633-762d-47cc-807a-c70afb7e0c83 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:05.383056 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:05.413670 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe69e7-5463-470a-9b9e-1f8c06be9ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:05.414326 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe69e7-5463-470a-9b9e-1f8c06be9ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:05.414607 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe69e7-5463-470a-9b9e-1f8c06be9ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:05.414878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe69e7-5463-470a-9b9e-1f8c06be9ff7 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:05.415550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe69e7-5463-470a-9b9e-1f8c06be9ff7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:05.415849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe69e7-5463-470a-9b9e-1f8c06be9ff7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:05.417299 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe69e7-5463-470a-9b9e-1f8c06be9ff7 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:05.428143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 210/419] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:01:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:05.434010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc139dc-e165-4fc1-bf93-46630eca7c14 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:05.512802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1a8c7-ea1e-45d9-9a6a-cbbcefb60e83 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:05.513158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1a8c7-ea1e-45d9-9a6a-cbbcefb60e83 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:05.513355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1a8c7-ea1e-45d9-9a6a-cbbcefb60e83 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:05.515396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 211/420] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:05.525136 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2361a8b-a5d0-428b-9e2a-aa6fbaa9dad8 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:05.525421 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2361a8b-a5d0-428b-9e2a-aa6fbaa9dad8 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:05.525706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2361a8b-a5d0-428b-9e2a-aa6fbaa9dad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:05.543061 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc139dc-e165-4fc1-bf93-46630eca7c14 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:05.544384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-a2361a8b-a5d0-428b-9e2a-aa6fbaa9dad8 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:05.545645 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a2361a8b-a5d0-428b-9e2a-aa6fbaa9dad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:05.577331 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:05.587330 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f3aea-d34f-46ac-9d64-75721e4d8cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:05.587947 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f3aea-d34f-46ac-9d64-75721e4d8cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:05.588294 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f3aea-d34f-46ac-9d64-75721e4d8cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:05.588575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f3aea-d34f-46ac-9d64-75721e4d8cf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:05.632270 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 212/421] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:01:05.717713 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f3aea-d34f-46ac-9d64-75721e4d8cf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:05.719194 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f3aea-d34f-46ac-9d64-75721e4d8cf8 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:05.864246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 210/422] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:05.869972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2361a8b-a5d0-428b-9e2a-aa6fbaa9dad8 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:05.982472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83626d-5bdf-4810-bf8f-891fa37df36a None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:05.982816 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83626d-5bdf-4810-bf8f-891fa37df36a None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:05.983036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83626d-5bdf-4810-bf8f-891fa37df36a None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:05.984662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 211/423] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:01:05.989250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b214f0-08fb-4165-960f-cf92d6e389ac None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:05.989533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b214f0-08fb-4165-960f-cf92d6e389ac None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:05.990517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b214f0-08fb-4165-960f-cf92d6e389ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:05.993460 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2361a8b-a5d0-428b-9e2a-aa6fbaa9dad8 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:06.001353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:06.029512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4bcd98-ee7f-466d-9087-1f8bc502b069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:06.030132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bcd98-ee7f-466d-9087-1f8bc502b069 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:06.030410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bcd98-ee7f-466d-9087-1f8bc502b069 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:06.030682 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bcd98-ee7f-466d-9087-1f8bc502b069 None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:06.031792 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4bcd98-ee7f-466d-9087-1f8bc502b069 None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:06.033367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4bcd98-ee7f-466d-9087-1f8bc502b069 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:06.035258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 213/424] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:01:05 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:06.037563 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-39b214f0-08fb-4165-960f-cf92d6e389ac None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:06.038831 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-39b214f0-08fb-4165-960f-cf92d6e389ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:06.041614 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4bcd98-ee7f-466d-9087-1f8bc502b069 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:06.080598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:06.156798 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 212/425] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:06.157669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b214f0-08fb-4165-960f-cf92d6e389ac None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:06.163216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4bcd98-ee7f-466d-9087-1f8bc502b069 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:06.209628 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a5cbf-9a69-48db-8499-ed56a226f153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:06.210542 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a5cbf-9a69-48db-8499-ed56a226f153 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:06.210829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a5cbf-9a69-48db-8499-ed56a226f153 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:06.211107 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a5cbf-9a69-48db-8499-ed56a226f153 None admin] PATH_INFO: `/v3/groups/8b050f7eb7bb48f4bc556655ec209b61` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:06.212415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a5cbf-9a69-48db-8499-ed56a226f153 None admin] RBAC: Authorizing `identity:get_group(group_id=8b050f7eb7bb48f4bc556655ec209b61)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:06.213906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a5cbf-9a69-48db-8499-ed56a226f153 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:06.215784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 214/426] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 12 12:01:06 2022] GET /identity/v3/groups/8b050f7eb7bb48f4bc556655ec209b61 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:06.222561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a5cbf-9a69-48db-8499-ed56a226f153 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:06.317463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b214f0-08fb-4165-960f-cf92d6e389ac None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:06.319445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:06.355090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131e1e9-7611-4d42-9df1-f12e56102cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:06.356408 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131e1e9-7611-4d42-9df1-f12e56102cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:06.356726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131e1e9-7611-4d42-9df1-f12e56102cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:06.357158 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131e1e9-7611-4d42-9df1-f12e56102cb1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:06.369607 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131e1e9-7611-4d42-9df1-f12e56102cb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-c131e1e9-7611-4d42-9df1-f12e56102cb1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:06.375012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:06.375877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:06.377070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:06.377070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:06.377070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:06.377070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:06.377070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:06.377070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:06.377070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:06.377070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:06.377070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:06.377070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:06.377070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 12 12:01:06.377070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:06.377070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 213/427] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:06.383008 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131e1e9-7611-4d42-9df1-f12e56102cb1 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:06.417054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:06.496659 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131e1e9-7611-4d42-9df1-f12e56102cb1 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:06.512333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a5cbf-9a69-48db-8499-ed56a226f153 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:06.538108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff12b9-7bd0-4120-b20c-6d5d20fcd26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:06.538722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff12b9-7bd0-4120-b20c-6d5d20fcd26c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:06.539005 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff12b9-7bd0-4120-b20c-6d5d20fcd26c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:06.539279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff12b9-7bd0-4120-b20c-6d5d20fcd26c None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:06.539955 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff12b9-7bd0-4120-b20c-6d5d20fcd26c None admin] RBAC: Adding query filter params (name=member) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:06.540394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff12b9-7bd0-4120-b20c-6d5d20fcd26c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:06.541826 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff12b9-7bd0-4120-b20c-6d5d20fcd26c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:06.552303 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 214/428] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:01:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:06.557595 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff12b9-7bd0-4120-b20c-6d5d20fcd26c None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:06.582113 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509e136-2502-4f76-bfae-010628ed2558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:06.583592 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509e136-2502-4f76-bfae-010628ed2558 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:06.584374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509e136-2502-4f76-bfae-010628ed2558 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:06.584893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509e136-2502-4f76-bfae-010628ed2558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:06.586022 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509e136-2502-4f76-bfae-010628ed2558 None admin] RBAC: Adding query filter params (group.id=8b050f7eb7bb48f4bc556655ec209b61, role.id=8b2ee9b11a934b67aa3fd68c497fbdf7, scope.project.id=4fe97b975d834d83a6733a0b2591bba4) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:06.586839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509e136-2502-4f76-bfae-010628ed2558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:06.588914 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509e136-2502-4f76-bfae-010628ed2558 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:06.597422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 215/429] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 12 12:01:06 2022] GET /identity/v3/role_assignments?group.id=8b050f7eb7bb48f4bc556655ec209b61&scope.project.id=4fe97b975d834d83a6733a0b2591bba4&role.id=8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 611 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:06.598425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509e136-2502-4f76-bfae-010628ed2558 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:06.689790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:06.724793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509e136-2502-4f76-bfae-010628ed2558 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:06.734790 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:06.741829 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff12b9-7bd0-4120-b20c-6d5d20fcd26c None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:06.769469 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f32b8a-2913-4902-987e-1a39d8579efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:06.770378 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f32b8a-2913-4902-987e-1a39d8579efe None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:06.771548 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f32b8a-2913-4902-987e-1a39d8579efe None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:06.772031 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f32b8a-2913-4902-987e-1a39d8579efe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:06.840292 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9c643-3dee-4f11-ba92-6227e8f0cb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:06.840985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9c643-3dee-4f11-ba92-6227e8f0cb1d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:06.841635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9c643-3dee-4f11-ba92-6227e8f0cb1d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:06.841635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9c643-3dee-4f11-ba92-6227e8f0cb1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:06.862532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9c643-3dee-4f11-ba92-6227e8f0cb1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:06.867480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f32b8a-2913-4902-987e-1a39d8579efe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-87e9c643-3dee-4f11-ba92-6227e8f0cb1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:06.868522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:06.869716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:06.870526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:06.870526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:06.870526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:06.870526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:06.870526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:06.870526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:06.870526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:06.870526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:06.870526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:06.870526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:06.870526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:01:06.870526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:06.870526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 215/430] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:01:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:06.872208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f32b8a-2913-4902-987e-1a39d8579efe None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:06.876404 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9c643-3dee-4f11-ba92-6227e8f0cb1d None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:07.001495 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 216/431] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:07.008486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f32b8a-2913-4902-987e-1a39d8579efe None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:07.009729 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9c643-3dee-4f11-ba92-6227e8f0cb1d None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:07.017240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:07.067475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3129a97-85a7-458a-ae42-8153a5197544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:07.068858 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3129a97-85a7-458a-ae42-8153a5197544 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:07.069369 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3129a97-85a7-458a-ae42-8153a5197544 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:07.069744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3129a97-85a7-458a-ae42-8153a5197544 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:07.070432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3129a97-85a7-458a-ae42-8153a5197544 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:07.070739 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3129a97-85a7-458a-ae42-8153a5197544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:07.072480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3129a97-85a7-458a-ae42-8153a5197544 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:07.080050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:07.084746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 216/432] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:07.096860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3129a97-85a7-458a-ae42-8153a5197544 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:07.148970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f32b8a-2913-4902-987e-1a39d8579efe None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:07.190730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124c603-2056-4120-a2ca-85258bec4e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:07.191204 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124c603-2056-4120-a2ca-85258bec4e2a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:07.191481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124c603-2056-4120-a2ca-85258bec4e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:07.191754 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124c603-2056-4120-a2ca-85258bec4e2a None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:07.193448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7124c603-2056-4120-a2ca-85258bec4e2a None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:07.195085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7124c603-2056-4120-a2ca-85258bec4e2a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:07.197027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 217/433] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:01:07 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:07.204703 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124c603-2056-4120-a2ca-85258bec4e2a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:07.239897 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3129a97-85a7-458a-ae42-8153a5197544 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:07.279923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc38e9-8e55-4164-a7c3-632d78d57a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:07.280717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc38e9-8e55-4164-a7c3-632d78d57a87 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:07.281060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc38e9-8e55-4164-a7c3-632d78d57a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:07.281453 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc38e9-8e55-4164-a7c3-632d78d57a87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:07.296321 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:07.332922 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124c603-2056-4120-a2ca-85258bec4e2a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:07.368504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:07.372149 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda54e3-6aca-4551-8774-12e1235f8f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:07.372814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda54e3-6aca-4551-8774-12e1235f8f9e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:07.373096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda54e3-6aca-4551-8774-12e1235f8f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:07.373365 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda54e3-6aca-4551-8774-12e1235f8f9e None admin] PATH_INFO: `/v3/groups/8b050f7eb7bb48f4bc556655ec209b61` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:07.373616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc38e9-8e55-4164-a7c3-632d78d57a87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:07.374895 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda54e3-6aca-4551-8774-12e1235f8f9e None admin] RBAC: Authorizing `identity:get_group(group_id=8b050f7eb7bb48f4bc556655ec209b61)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:07.375086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc38e9-8e55-4164-a7c3-632d78d57a87 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:07.376534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda54e3-6aca-4551-8774-12e1235f8f9e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:07.378590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 218/434] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 12 12:01:07 2022] GET /identity/v3/groups/8b050f7eb7bb48f4bc556655ec209b61 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:07.379437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed038ab4-173a-4b2f-b14a-ecb19c2291e1 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:07.380724 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed038ab4-173a-4b2f-b14a-ecb19c2291e1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:07.380724 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed038ab4-173a-4b2f-b14a-ecb19c2291e1 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:07.382258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 219/435] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:07.384447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed038ab4-173a-4b2f-b14a-ecb19c2291e1 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:07.491381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 217/436] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:07.492398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed038ab4-173a-4b2f-b14a-ecb19c2291e1 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:07.492671 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d864177-ed6d-4e54-8c3c-2993309f392a None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:07.492954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d864177-ed6d-4e54-8c3c-2993309f392a None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:07.493230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d864177-ed6d-4e54-8c3c-2993309f392a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:07.511206 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-8d864177-ed6d-4e54-8c3c-2993309f392a None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:07.512404 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8d864177-ed6d-4e54-8c3c-2993309f392a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:07.529088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99555bf-1fd1-4b89-8fd0-d8f0c1bfe6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:07.529683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99555bf-1fd1-4b89-8fd0-d8f0c1bfe6fe None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:07.529958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99555bf-1fd1-4b89-8fd0-d8f0c1bfe6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:07.530224 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99555bf-1fd1-4b89-8fd0-d8f0c1bfe6fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:07.530939 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99555bf-1fd1-4b89-8fd0-d8f0c1bfe6fe None admin] RBAC: Adding query filter params (group.id=8b050f7eb7bb48f4bc556655ec209b61, role.id=8b2ee9b11a934b67aa3fd68c497fbdf7, scope.project.id=ad5d2677b95b428a93e499f4ced1b86b) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:07.531240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99555bf-1fd1-4b89-8fd0-d8f0c1bfe6fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:07.532695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99555bf-1fd1-4b89-8fd0-d8f0c1bfe6fe None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:07.538823 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 220/437] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 12 12:01:07 2022] GET /identity/v3/role_assignments?group.id=8b050f7eb7bb48f4bc556655ec209b61&scope.project.id=ad5d2677b95b428a93e499f4ced1b86b&role.id=8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:07.540024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dfd68-c751-4071-8287-ad6e5ceb60dc None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:07.540293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dfd68-c751-4071-8287-ad6e5ceb60dc None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:07.540565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dfd68-c751-4071-8287-ad6e5ceb60dc None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:07.541783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 221/438] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:07.544919 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dfd68-c751-4071-8287-ad6e5ceb60dc None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:07.556187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:07.595718 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 218/439] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:07.597280 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8606b-6ffb-4b9e-9cc6-ebab579da388 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:07.597569 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8606b-6ffb-4b9e-9cc6-ebab579da388 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:07.597843 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8606b-6ffb-4b9e-9cc6-ebab579da388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:07.618234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-9ea8606b-6ffb-4b9e-9cc6-ebab579da388 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:07.619535 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9ea8606b-6ffb-4b9e-9cc6-ebab579da388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:07.647105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:07.690017 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dfd68-c751-4071-8287-ad6e5ceb60dc None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:07.701179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 219/440] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:07.741521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0634d9-5b28-4427-a9a7-9f90aecc559f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:07.742176 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0634d9-5b28-4427-a9a7-9f90aecc559f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:07.742469 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0634d9-5b28-4427-a9a7-9f90aecc559f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:07.742749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0634d9-5b28-4427-a9a7-9f90aecc559f None admin] PATH_INFO: `/v3/projects/4fe97b975d834d83a6733a0b2591bba4` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:07.743918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0634d9-5b28-4427-a9a7-9f90aecc559f None admin] RBAC: Authorizing `identity:get_project(project_id=4fe97b975d834d83a6733a0b2591bba4)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:07.745438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0634d9-5b28-4427-a9a7-9f90aecc559f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:07.747271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 222/441] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:01:07 2022] GET /identity/v3/projects/4fe97b975d834d83a6733a0b2591bba4 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:07.753270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8606b-6ffb-4b9e-9cc6-ebab579da388 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:07.862428 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8606b-6ffb-4b9e-9cc6-ebab579da388 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:07.892401 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0634d9-5b28-4427-a9a7-9f90aecc559f None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:07.898520 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c072aa-c15b-4726-97dc-ba56bd18a3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:07.899158 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c072aa-c15b-4726-97dc-ba56bd18a3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:07.899438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c072aa-c15b-4726-97dc-ba56bd18a3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:07.899707 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c072aa-c15b-4726-97dc-ba56bd18a3d4 None admin] PATH_INFO: `/v3/groups/8b050f7eb7bb48f4bc556655ec209b61` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:07.900850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c072aa-c15b-4726-97dc-ba56bd18a3d4 None admin] RBAC: Authorizing `identity:get_group(group_id=8b050f7eb7bb48f4bc556655ec209b61)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:07.902726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c072aa-c15b-4726-97dc-ba56bd18a3d4 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:07.904849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 220/442] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 12 12:01:07 2022] GET /identity/v3/groups/8b050f7eb7bb48f4bc556655ec209b61 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:07.910343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c072aa-c15b-4726-97dc-ba56bd18a3d4 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:07.940836 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:07.976489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:08.000582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0634d9-5b28-4427-a9a7-9f90aecc559f None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:08.012587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c072aa-c15b-4726-97dc-ba56bd18a3d4 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:08.034408 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46924ea-0f5e-4a49-ba63-e5e45cd21871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:08.035019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46924ea-0f5e-4a49-ba63-e5e45cd21871 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:08.035299 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46924ea-0f5e-4a49-ba63-e5e45cd21871 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:08.035745 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46924ea-0f5e-4a49-ba63-e5e45cd21871 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:08.046770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfcaefd-0c4f-4718-892e-d93655056f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:08.047319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfcaefd-0c4f-4718-892e-d93655056f2a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:08.052818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46924ea-0f5e-4a49-ba63-e5e45cd21871 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:08.052976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfcaefd-0c4f-4718-892e-d93655056f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:08.052976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfcaefd-0c4f-4718-892e-d93655056f2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:08.052976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfcaefd-0c4f-4718-892e-d93655056f2a None admin] RBAC: Adding query filter params (group.id=8b050f7eb7bb48f4bc556655ec209b61, role.id=b5e411fc96434df69c6cc1df951b3e6f, scope.project.id=4fe97b975d834d83a6733a0b2591bba4) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:08.052976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfcaefd-0c4f-4718-892e-d93655056f2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:08.052976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfcaefd-0c4f-4718-892e-d93655056f2a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-c46924ea-0f5e-4a49-ba63-e5e45cd21871 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:08.053423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:08.054438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:08.055278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:08.055278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:08.055278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:08.055278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:08.055278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:08.055278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:08.055278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:08.055278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:08.055278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:08.055278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:08.055278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:01:08.055278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:08.055278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 223/443] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:01:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:01:08.060410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46924ea-0f5e-4a49-ba63-e5e45cd21871 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:08.060694 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 221/444] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 12 12:01:07 2022] GET /identity/v3/role_assignments?group.id=8b050f7eb7bb48f4bc556655ec209b61&scope.project.id=4fe97b975d834d83a6733a0b2591bba4&role.id=b5e411fc96434df69c6cc1df951b3e6f => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:08.061634 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfcaefd-0c4f-4718-892e-d93655056f2a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:08.169034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:08.177140 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46924ea-0f5e-4a49-ba63-e5e45cd21871 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:08.179160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfcaefd-0c4f-4718-892e-d93655056f2a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:08.221088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2cf13-cf75-4d50-92d8-d4aecb65ee54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:08.221810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2cf13-cf75-4d50-92d8-d4aecb65ee54 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:08.222095 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2cf13-cf75-4d50-92d8-d4aecb65ee54 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:08.222369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2cf13-cf75-4d50-92d8-d4aecb65ee54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:08.224500 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe30d3-19c2-4048-8467-3534c5ecd269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:08.225152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:08.226609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe30d3-19c2-4048-8467-3534c5ecd269 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:08.226609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe30d3-19c2-4048-8467-3534c5ecd269 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:08.228043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe30d3-19c2-4048-8467-3534c5ecd269 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:08.228043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe30d3-19c2-4048-8467-3534c5ecd269 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:08.228043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe30d3-19c2-4048-8467-3534c5ecd269 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:08.230655 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe30d3-19c2-4048-8467-3534c5ecd269 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:08.240781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 222/445] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:08.241623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2cf13-cf75-4d50-92d8-d4aecb65ee54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-dbd2cf13-cf75-4d50-92d8-d4aecb65ee54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:08.246700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:08.247612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:08.248390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:08.248390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:08.248390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:08.248390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:08.248390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:08.248390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:08.248390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:08.248390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:08.248390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:08.248390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:08.248390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:08.248390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:08.248390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 224/446] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:01:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:08.249718 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe30d3-19c2-4048-8467-3534c5ecd269 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:08.254175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2cf13-cf75-4d50-92d8-d4aecb65ee54 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:08.372398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2cf13-cf75-4d50-92d8-d4aecb65ee54 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:08.387466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe30d3-19c2-4048-8467-3534c5ecd269 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:08.413329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c62c4-e2b6-4c12-8fb6-3a9e8a536f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:08.413937 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c62c4-e2b6-4c12-8fb6-3a9e8a536f78 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:08.414212 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c62c4-e2b6-4c12-8fb6-3a9e8a536f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:08.414485 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c62c4-e2b6-4c12-8fb6-3a9e8a536f78 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:08.415148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c62c4-e2b6-4c12-8fb6-3a9e8a536f78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:08.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c62c4-e2b6-4c12-8fb6-3a9e8a536f78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:08.416879 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c62c4-e2b6-4c12-8fb6-3a9e8a536f78 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:08.426129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 225/447] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:08.432289 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8c385-f1a0-46a4-b825-cdb3678c558e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:08.432576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c62c4-e2b6-4c12-8fb6-3a9e8a536f78 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:08.433046 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8c385-f1a0-46a4-b825-cdb3678c558e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:08.433263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8c385-f1a0-46a4-b825-cdb3678c558e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:08.433536 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8c385-f1a0-46a4-b825-cdb3678c558e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:08.465369 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:08.501411 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:08.517452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8c385-f1a0-46a4-b825-cdb3678c558e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:08.518919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8c385-f1a0-46a4-b825-cdb3678c558e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:08.550408 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c62c4-e2b6-4c12-8fb6-3a9e8a536f78 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:08.590981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0c4c4-6528-4edc-9a62-a53e91a297ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:08.591577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0c4c4-6528-4edc-9a62-a53e91a297ac None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:08.591859 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0c4c4-6528-4edc-9a62-a53e91a297ac None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:08.592127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0c4c4-6528-4edc-9a62-a53e91a297ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:08.628011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 223/448] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:08.634339 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8c385-f1a0-46a4-b825-cdb3678c558e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:08.660439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0c4c4-6528-4edc-9a62-a53e91a297ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:08.662567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0c4c4-6528-4edc-9a62-a53e91a297ac None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:08.688236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:08.717018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:08.732260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8c385-f1a0-46a4-b825-cdb3678c558e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:08.761242 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 226/449] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:08.766352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45d30d-29ea-41ca-9c43-2cc9759ea4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:08.766915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0c4c4-6528-4edc-9a62-a53e91a297ac None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:08.767011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45d30d-29ea-41ca-9c43-2cc9759ea4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:08.767233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45d30d-29ea-41ca-9c43-2cc9759ea4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:08.767502 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45d30d-29ea-41ca-9c43-2cc9759ea4e4 None admin] PATH_INFO: `/v3/projects/ad5d2677b95b428a93e499f4ced1b86b` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:08.768602 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45d30d-29ea-41ca-9c43-2cc9759ea4e4 None admin] RBAC: Authorizing `identity:get_project(project_id=ad5d2677b95b428a93e499f4ced1b86b)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:08.770049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45d30d-29ea-41ca-9c43-2cc9759ea4e4 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:08.771855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 224/450] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:01:08 2022] GET /identity/v3/projects/ad5d2677b95b428a93e499f4ced1b86b => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:08.777073 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45d30d-29ea-41ca-9c43-2cc9759ea4e4 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:08.869161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0c4c4-6528-4edc-9a62-a53e91a297ac None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:08.874800 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45d30d-29ea-41ca-9c43-2cc9759ea4e4 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:08.898264 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:08.903842 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016d76d-a818-41be-9068-dcae9a50bb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:08.904463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016d76d-a818-41be-9068-dcae9a50bb75 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:08.904774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016d76d-a818-41be-9068-dcae9a50bb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:08.905053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016d76d-a818-41be-9068-dcae9a50bb75 None admin] PATH_INFO: `/v3/projects/9e93c7bb262f46249ca80b95c794bf2f` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:08.906161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016d76d-a818-41be-9068-dcae9a50bb75 None admin] RBAC: Authorizing `identity:get_project(project_id=9e93c7bb262f46249ca80b95c794bf2f)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:08.907629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016d76d-a818-41be-9068-dcae9a50bb75 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:08.909040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2b452-1e30-40c9-aec6-c665b8d84e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:08.909490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 227/451] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:01:08 2022] GET /identity/v3/projects/9e93c7bb262f46249ca80b95c794bf2f => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:08.909668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2b452-1e30-40c9-aec6-c665b8d84e6b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:08.909952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2b452-1e30-40c9-aec6-c665b8d84e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:08.910228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2b452-1e30-40c9-aec6-c665b8d84e6b None admin] PATH_INFO: `/v3/groups/8b050f7eb7bb48f4bc556655ec209b61` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:08.911423 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2b452-1e30-40c9-aec6-c665b8d84e6b None admin] RBAC: Authorizing `identity:get_group(group_id=8b050f7eb7bb48f4bc556655ec209b61)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:08.913088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2b452-1e30-40c9-aec6-c665b8d84e6b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:08.919196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 225/452] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 12 12:01:08 2022] GET /identity/v3/groups/8b050f7eb7bb48f4bc556655ec209b61 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:08.919326 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016d76d-a818-41be-9068-dcae9a50bb75 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:08.920324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2b452-1e30-40c9-aec6-c665b8d84e6b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:08.939577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:09.012731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016d76d-a818-41be-9068-dcae9a50bb75 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:09.017650 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2b452-1e30-40c9-aec6-c665b8d84e6b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:09.046960 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcea100-f29b-4420-8f73-c3e85d75f912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:09.047602 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcea100-f29b-4420-8f73-c3e85d75f912 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:09.047887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcea100-f29b-4420-8f73-c3e85d75f912 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:09.048159 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcea100-f29b-4420-8f73-c3e85d75f912 None admin] PATH_INFO: `/v3/groups/23bd955784184c7488dd4fed6e261c0a` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:09.049306 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcea100-f29b-4420-8f73-c3e85d75f912 None admin] RBAC: Authorizing `identity:get_group(group_id=23bd955784184c7488dd4fed6e261c0a)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:09.050481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a25af-1adf-49bf-85dd-f33f0b841057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:09.050772 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcea100-f29b-4420-8f73-c3e85d75f912 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:09.051071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a25af-1adf-49bf-85dd-f33f0b841057 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:09.051349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a25af-1adf-49bf-85dd-f33f0b841057 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:09.051616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a25af-1adf-49bf-85dd-f33f0b841057 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:09.052866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a25af-1adf-49bf-85dd-f33f0b841057 None admin] RBAC: Adding query filter params (group.id=8b050f7eb7bb48f4bc556655ec209b61, role.id=b5e411fc96434df69c6cc1df951b3e6f, scope.project.id=ad5d2677b95b428a93e499f4ced1b86b) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:09.058133 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 228/453] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 12 12:01:08 2022] GET /identity/v3/groups/23bd955784184c7488dd4fed6e261c0a => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:09.058280 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a25af-1adf-49bf-85dd-f33f0b841057 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:09.058280 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a25af-1adf-49bf-85dd-f33f0b841057 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:09.059079 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcea100-f29b-4420-8f73-c3e85d75f912 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:09.070263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 226/454] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 12 12:01:08 2022] GET /identity/v3/role_assignments?group.id=8b050f7eb7bb48f4bc556655ec209b61&scope.project.id=ad5d2677b95b428a93e499f4ced1b86b&role.id=b5e411fc96434df69c6cc1df951b3e6f => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:09.154768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:09.163767 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcea100-f29b-4420-8f73-c3e85d75f912 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:09.199707 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18156ef0-73cb-4c8a-b965-9da7da6892ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:09.200237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18156ef0-73cb-4c8a-b965-9da7da6892ab None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:09.200519 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18156ef0-73cb-4c8a-b965-9da7da6892ab None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:09.200829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18156ef0-73cb-4c8a-b965-9da7da6892ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:09.201559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18156ef0-73cb-4c8a-b965-9da7da6892ab None admin] RBAC: Adding query filter params (group.id=23bd955784184c7488dd4fed6e261c0a, role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=9e93c7bb262f46249ca80b95c794bf2f) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:09.201868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18156ef0-73cb-4c8a-b965-9da7da6892ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:09.203312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18156ef0-73cb-4c8a-b965-9da7da6892ab None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:09.209894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 229/455] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 12 12:01:09 2022] GET /identity/v3/role_assignments?group.id=23bd955784184c7488dd4fed6e261c0a&scope.project.id=9e93c7bb262f46249ca80b95c794bf2f&role.id=906e3a8a6c7540a38f629f2567133347 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:11.482605 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8f452-3fe0-46ec-bbdb-7c2e53113d00 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:11.483129 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8f452-3fe0-46ec-bbdb-7c2e53113d00 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:11.483129 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8f452-3fe0-46ec-bbdb-7c2e53113d00 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:11.484834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 227/456] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:11.491104 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879c18d-f5b2-4bb8-80da-067d86c2ac1b None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:11.491561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879c18d-f5b2-4bb8-80da-067d86c2ac1b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:11.491733 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879c18d-f5b2-4bb8-80da-067d86c2ac1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:11.513112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f879c18d-f5b2-4bb8-80da-067d86c2ac1b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:11.514303 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f879c18d-f5b2-4bb8-80da-067d86c2ac1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:11.597410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 230/457] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:11.830000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea0554-ce13-4105-8e8e-d4a90d02b8aa None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:11.830000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea0554-ce13-4105-8e8e-d4a90d02b8aa None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:11.830000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea0554-ce13-4105-8e8e-d4a90d02b8aa None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:11.830648 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 228/458] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:11.835368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef60dd-4832-48f6-adc5-ae002311b324 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:11.835657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef60dd-4832-48f6-adc5-ae002311b324 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:11.835936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef60dd-4832-48f6-adc5-ae002311b324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:11.857193 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9aef60dd-4832-48f6-adc5-ae002311b324 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:11.858379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9aef60dd-4832-48f6-adc5-ae002311b324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:11.943356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea0554-ce13-4105-8e8e-d4a90d02b8aa None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:11.961437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 231/459] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:12.014119 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:12.020192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9141d00-1059-4bc6-925e-f9e8a2384071 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:12.021059 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9141d00-1059-4bc6-925e-f9e8a2384071 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:12.021339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9141d00-1059-4bc6-925e-f9e8a2384071 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:12.022984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 232/460] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:12.027948 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9316e48-3744-435b-97ef-a8cc9aacbfef None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:12.028468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9316e48-3744-435b-97ef-a8cc9aacbfef None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:12.028992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9316e48-3744-435b-97ef-a8cc9aacbfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:12.064428 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-e9316e48-3744-435b-97ef-a8cc9aacbfef None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:12.065963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e9316e48-3744-435b-97ef-a8cc9aacbfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:12.078310 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea0554-ce13-4105-8e8e-d4a90d02b8aa None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:12.081607 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:12.154172 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65abe8c7-4620-469a-a0b7-f79027dd1bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:12.155159 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65abe8c7-4620-469a-a0b7-f79027dd1bef None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:12.155792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65abe8c7-4620-469a-a0b7-f79027dd1bef None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:12.157325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65abe8c7-4620-469a-a0b7-f79027dd1bef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:12.172454 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 233/461] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 12 12:01:12.185784 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65abe8c7-4620-469a-a0b7-f79027dd1bef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:12.187604 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65abe8c7-4620-469a-a0b7-f79027dd1bef None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-65abe8c7-4620-469a-a0b7-f79027dd1bef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:12.209072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:12.209993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:12.211431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:12.211431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:12.211431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:12.211431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 229/462] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:12.216031 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9316e48-3744-435b-97ef-a8cc9aacbfef None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:12.317276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65abe8c7-4620-469a-a0b7-f79027dd1bef None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:12.374512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9316e48-3744-435b-97ef-a8cc9aacbfef None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:12.410275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991da91-f4ce-42a7-bf73-41f0daf977cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:12.411799 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991da91-f4ce-42a7-bf73-41f0daf977cc None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:12.412319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991da91-f4ce-42a7-bf73-41f0daf977cc None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:12.413040 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991da91-f4ce-42a7-bf73-41f0daf977cc None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:12.413951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991da91-f4ce-42a7-bf73-41f0daf977cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:12.414477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991da91-f4ce-42a7-bf73-41f0daf977cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:12.416046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991da91-f4ce-42a7-bf73-41f0daf977cc None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:12.428888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 234/463] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:12.434906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991da91-f4ce-42a7-bf73-41f0daf977cc None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:12.447637 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:12.514096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65abe8c7-4620-469a-a0b7-f79027dd1bef None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:12.521851 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:12.549087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dffaa5-5967-44fe-860d-7ab1ff360c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:12.550228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dffaa5-5967-44fe-860d-7ab1ff360c68 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:12.550961 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dffaa5-5967-44fe-860d-7ab1ff360c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:12.551243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dffaa5-5967-44fe-860d-7ab1ff360c68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:12.563024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dffaa5-5967-44fe-860d-7ab1ff360c68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:12.564685 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dffaa5-5967-44fe-860d-7ab1ff360c68 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:12.565366 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991da91-f4ce-42a7-bf73-41f0daf977cc None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-87dffaa5-5967-44fe-860d-7ab1ff360c68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:12.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:12.580528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:12.582051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:12.582051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:12.582051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:12.582051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 230/464] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:12.585860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dffaa5-5967-44fe-860d-7ab1ff360c68 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:12.604387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fb5dc-4bfa-457b-993f-fc90c598668e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:12.605251 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fb5dc-4bfa-457b-993f-fc90c598668e None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:12.605677 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fb5dc-4bfa-457b-993f-fc90c598668e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:12.605950 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fb5dc-4bfa-457b-993f-fc90c598668e None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:12.606583 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1fb5dc-4bfa-457b-993f-fc90c598668e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:12.608011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1fb5dc-4bfa-457b-993f-fc90c598668e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:12.685526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dffaa5-5967-44fe-860d-7ab1ff360c68 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:12.691733 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 235/465] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:01:12 2022] POST /identity/v3/users => generated 246 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:01:12.694041 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6a6ba-c275-4754-9e82-5d9456b8266a None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:12.694399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6a6ba-c275-4754-9e82-5d9456b8266a None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:12.694782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6a6ba-c275-4754-9e82-5d9456b8266a None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:12.696370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 236/466] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:12.701690 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6a6ba-c275-4754-9e82-5d9456b8266a None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:12.738330 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821e6df-4f55-4425-953f-c3253fee6fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:12.739021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821e6df-4f55-4425-953f-c3253fee6fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:12.739852 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821e6df-4f55-4425-953f-c3253fee6fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:12.740515 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821e6df-4f55-4425-953f-c3253fee6fc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:12.756744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821e6df-4f55-4425-953f-c3253fee6fc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:12.758554 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821e6df-4f55-4425-953f-c3253fee6fc2 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-0821e6df-4f55-4425-953f-c3253fee6fc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:12.780296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:12.781256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:12.782054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:12.782054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:12.782054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:12.782054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 231/467] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:12.782054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb99bc8-e64d-4d3a-8d53-2d6aa6cae9e7 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:12.782054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb99bc8-e64d-4d3a-8d53-2d6aa6cae9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:12.782054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb99bc8-e64d-4d3a-8d53-2d6aa6cae9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:12.817567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-cfb99bc8-e64d-4d3a-8d53-2d6aa6cae9e7 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:12.818991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-cfb99bc8-e64d-4d3a-8d53-2d6aa6cae9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:12.823232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:12.825297 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:12.889590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6a6ba-c275-4754-9e82-5d9456b8266a None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:12.913438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 232/468] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:12.914642 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb99bc8-e64d-4d3a-8d53-2d6aa6cae9e7 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:12.934649 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc17aec-ac96-4b87-b819-14598f7cc1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:12.935408 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc17aec-ac96-4b87-b819-14598f7cc1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:12.935883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc17aec-ac96-4b87-b819-14598f7cc1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:12.936219 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc17aec-ac96-4b87-b819-14598f7cc1a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:12.937381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc17aec-ac96-4b87-b819-14598f7cc1a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:12.937756 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc17aec-ac96-4b87-b819-14598f7cc1a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:12.939215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc17aec-ac96-4b87-b819-14598f7cc1a6 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:12.960728 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 237/469] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:12.973774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc17aec-ac96-4b87-b819-14598f7cc1a6 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:13.035555 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb99bc8-e64d-4d3a-8d53-2d6aa6cae9e7 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:13.074479 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfa9fb-6ae9-4067-bb30-a7f5906c2441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:13.075353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfa9fb-6ae9-4067-bb30-a7f5906c2441 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:13.075830 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfa9fb-6ae9-4067-bb30-a7f5906c2441 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:13.076261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfa9fb-6ae9-4067-bb30-a7f5906c2441 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:13.077067 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfa9fb-6ae9-4067-bb30-a7f5906c2441 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:13.077526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfa9fb-6ae9-4067-bb30-a7f5906c2441 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:13.079220 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfa9fb-6ae9-4067-bb30-a7f5906c2441 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:13.088757 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:13.092964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 233/470] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:13.098653 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfa9fb-6ae9-4067-bb30-a7f5906c2441 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:13.142671 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc17aec-ac96-4b87-b819-14598f7cc1a6 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:13.168349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:13.181237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87214a7a-8c71-4941-b076-db902c37a1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:13.181836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87214a7a-8c71-4941-b076-db902c37a1b3 None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:13.182117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87214a7a-8c71-4941-b076-db902c37a1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:13.182387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87214a7a-8c71-4941-b076-db902c37a1b3 None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:13.183023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87214a7a-8c71-4941-b076-db902c37a1b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:13.184705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87214a7a-8c71-4941-b076-db902c37a1b3 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:13.226972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfa9fb-6ae9-4067-bb30-a7f5906c2441 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:13.262846 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c010cf6-8afd-4c52-8c2a-42b9a0bf812a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:13.263692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c010cf6-8afd-4c52-8c2a-42b9a0bf812a None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:13.263977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c010cf6-8afd-4c52-8c2a-42b9a0bf812a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:13.264250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c010cf6-8afd-4c52-8c2a-42b9a0bf812a None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:13.264931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c010cf6-8afd-4c52-8c2a-42b9a0bf812a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:13.266381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c010cf6-8afd-4c52-8c2a-42b9a0bf812a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:13.278265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 238/471] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:01:12 2022] POST /identity/v3/users => generated 248 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:01:13.279444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a0997-a346-4ef2-8508-7d853d522d88 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:13.279734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a0997-a346-4ef2-8508-7d853d522d88 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:13.280010 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a0997-a346-4ef2-8508-7d853d522d88 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:13.284843 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 239/472] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:13.284843 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a0997-a346-4ef2-8508-7d853d522d88 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:13.376421 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 234/473] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:01:13 2022] POST /identity/v3/users => generated 248 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:01:13.382595 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7fd27-0b3a-453f-bdad-741676c35a0b None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:13.382940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7fd27-0b3a-453f-bdad-741676c35a0b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:13.383284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7fd27-0b3a-453f-bdad-741676c35a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:13.417370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a0997-a346-4ef2-8508-7d853d522d88 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:13.434794 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-32e7fd27-0b3a-453f-bdad-741676c35a0b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:13.436204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-32e7fd27-0b3a-453f-bdad-741676c35a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:13.469488 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf5d1b-fae8-492e-81f1-ac01183b90dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:13.470765 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf5d1b-fae8-492e-81f1-ac01183b90dc None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:13.471730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf5d1b-fae8-492e-81f1-ac01183b90dc None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:13.472458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf5d1b-fae8-492e-81f1-ac01183b90dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:13.489762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf5d1b-fae8-492e-81f1-ac01183b90dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:13.492319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf5d1b-fae8-492e-81f1-ac01183b90dc None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:13.506869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-3fbf5d1b-fae8-492e-81f1-ac01183b90dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:13.515500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:13.516468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:13.525195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:13.525195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:13.525195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:13.525195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 240/474] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:13.525770 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf5d1b-fae8-492e-81f1-ac01183b90dc None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:13.533489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 235/475] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:13.562466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:13.635628 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf5d1b-fae8-492e-81f1-ac01183b90dc None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:13.672339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb126a6-d2a8-4556-9819-31123880052f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:13.673218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb126a6-d2a8-4556-9819-31123880052f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:13.673646 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb126a6-d2a8-4556-9819-31123880052f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:13.674060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb126a6-d2a8-4556-9819-31123880052f None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:13.674841 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb126a6-d2a8-4556-9819-31123880052f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:13.675302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb126a6-d2a8-4556-9819-31123880052f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:13.677630 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb126a6-d2a8-4556-9819-31123880052f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:13.696248 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 241/476] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:13.697888 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7fd27-0b3a-453f-bdad-741676c35a0b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:13.827114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7fd27-0b3a-453f-bdad-741676c35a0b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:13.854984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb126a6-d2a8-4556-9819-31123880052f None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:13.892688 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f0a01-8f6c-4e4e-bca4-a9d75775356f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:13.893598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f0a01-8f6c-4e4e-bca4-a9d75775356f None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:13.894029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f0a01-8f6c-4e4e-bca4-a9d75775356f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:13.894443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f0a01-8f6c-4e4e-bca4-a9d75775356f None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:13.895278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f0a01-8f6c-4e4e-bca4-a9d75775356f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:13.896873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f0a01-8f6c-4e4e-bca4-a9d75775356f None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:13.978573 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:13.985881 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 236/477] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:01:13 2022] POST /identity/v3/users => generated 249 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:01:14.003348 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb126a6-d2a8-4556-9819-31123880052f None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:14.044385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4dfa65-4a65-492a-b58f-00ab57787578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:14.045052 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dfa65-4a65-492a-b58f-00ab57787578 None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:14.045333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dfa65-4a65-492a-b58f-00ab57787578 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:14.045608 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dfa65-4a65-492a-b58f-00ab57787578 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:14.046282 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4dfa65-4a65-492a-b58f-00ab57787578 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:14.047754 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4dfa65-4a65-492a-b58f-00ab57787578 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:14.093749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.sql.core [None req-2a4dfa65-4a65-492a-b58f-00ab57787578 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 12 12:01:14.093749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 12 12:01:14.093749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [parameters: {'id': 'a2f8c7f52eed4cb688ac42fdbdbe87a0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 12 12:01:14.093749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81794) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 12 12:01:14.114200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-2a4dfa65-4a65-492a-b58f-00ab57787578 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 12 12:01:14.114200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:14.114200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 12 12:01:14.114200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 12 12:01:14.114200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 12 12:01:14.114200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 12 12:01:14.114200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 12 12:01:14.114200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application result = self._query(query) Aug 12 12:01:14.114200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 12 12:01:14.114200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application conn.query(q) Aug 12 12:01:14.114200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 12 12:01:14.114200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 12 12:01:14.114200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 12 12:01:14.114200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application result.read() Aug 12 12:01:14.114200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return ref.to_dict() Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 12 12:01:14.115451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application next(self.gen) Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application yield resource Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application next(self.gen) Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self.session.rollback() Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self.force_reraise() Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise self.value Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application session.commit() Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 12 12:01:14.116334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._prepare_impl() Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self.session.flush() Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._flush(objects) Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application compat.raise_( Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application flush_context.execute() Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rec.execute(self) Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 12 12:01:14.117192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 12 12:01:14.118068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application result = self._query(query) Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application conn.query(q) Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application result.read() Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 12 12:01:14.118889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [parameters: {'id': 'a2f8c7f52eed4cb688ac42fdbdbe87a0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:14.119662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 12 12:01:14.120892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:14.121698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 242/478] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Aug 12 12:01:13 2022] POST /identity/v3/roles => generated 144 bytes in 261 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:14.121761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f0a01-8f6c-4e4e-bca4-a9d75775356f None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:14.193113 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:14.265211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f0a01-8f6c-4e4e-bca4-a9d75775356f None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:14.302992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81504e0b-81ec-48b4-8206-24ca056543a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:14.303624 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81504e0b-81ec-48b4-8206-24ca056543a3 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:14.303902 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81504e0b-81ec-48b4-8206-24ca056543a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:14.304171 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81504e0b-81ec-48b4-8206-24ca056543a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:14.316193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81504e0b-81ec-48b4-8206-24ca056543a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-81504e0b-81ec-48b4-8206-24ca056543a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:14.321771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:14.324174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:14.326211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:14.326211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:14.326211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:14.326211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:14.326211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:14.326211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:14.326211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:14.326211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:14.326211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:14.326211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:14.326211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 12 12:01:14.326211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:14.326211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 237/479] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:01:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:14.332672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4dfa65-4a65-492a-b58f-00ab57787578 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:14.434422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4dfa65-4a65-492a-b58f-00ab57787578 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:14.470482 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3657a17-71e3-47c1-b395-0ebb3e8b8302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:14.471087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3657a17-71e3-47c1-b395-0ebb3e8b8302 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:14.471370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3657a17-71e3-47c1-b395-0ebb3e8b8302 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:14.471646 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3657a17-71e3-47c1-b395-0ebb3e8b8302 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:14.472313 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3657a17-71e3-47c1-b395-0ebb3e8b8302 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:14.472622 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3657a17-71e3-47c1-b395-0ebb3e8b8302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:14.474347 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3657a17-71e3-47c1-b395-0ebb3e8b8302 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:14.487657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 243/480] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:14.911546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f2ce8-ccb9-40ea-a735-81f58795e7ef None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:14.912216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f2ce8-ccb9-40ea-a735-81f58795e7ef None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:14.912216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f2ce8-ccb9-40ea-a735-81f58795e7ef None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:14.913697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 238/481] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:14.918772 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859111f5-759f-4dd8-b861-df4e41afdbca None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:14.919234 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859111f5-759f-4dd8-b861-df4e41afdbca None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:14.919386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859111f5-759f-4dd8-b861-df4e41afdbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:14.940844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-859111f5-759f-4dd8-b861-df4e41afdbca None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:14.942538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-859111f5-759f-4dd8-b861-df4e41afdbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:14.959078 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:15.043395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 244/482] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:15.346622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f2ce8-ccb9-40ea-a735-81f58795e7ef None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:15.480293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f2ce8-ccb9-40ea-a735-81f58795e7ef None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:15.481931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:15.522881 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9ceb7-3ad5-47a2-93d9-07c3918a2aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:15.523512 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9ceb7-3ad5-47a2-93d9-07c3918a2aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:15.523791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9ceb7-3ad5-47a2-93d9-07c3918a2aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:15.524064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9ceb7-3ad5-47a2-93d9-07c3918a2aa9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:15.538075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9ceb7-3ad5-47a2-93d9-07c3918a2aa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-32d9ceb7-3ad5-47a2-93d9-07c3918a2aa9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:15.542789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:15.543879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:15.545176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:15.545176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:15.545176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:15.545176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:15.545176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:15.545176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:15.545176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:15.545176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:15.545176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:15.545176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:15.545176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:15.545176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:15.545176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 239/483] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:15.549646 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859111f5-759f-4dd8-b861-df4e41afdbca None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:15.686207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f346918-21ea-4ead-8e8e-e24c184d8aa8 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:15.686495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f346918-21ea-4ead-8e8e-e24c184d8aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:15.686772 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f346918-21ea-4ead-8e8e-e24c184d8aa8 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:15.688392 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 240/484] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:15.706105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec96fe-9140-4169-a7eb-f21ef7187d3f None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:15.706394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec96fe-9140-4169-a7eb-f21ef7187d3f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:15.706676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec96fe-9140-4169-a7eb-f21ef7187d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:15.730409 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-72ec96fe-9140-4169-a7eb-f21ef7187d3f None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:15.731605 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-72ec96fe-9140-4169-a7eb-f21ef7187d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:15.748812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859111f5-759f-4dd8-b861-df4e41afdbca None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:15.795524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48245e-b5b4-4e40-8723-76a29dbab81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:15.796449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48245e-b5b4-4e40-8723-76a29dbab81a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:15.796928 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48245e-b5b4-4e40-8723-76a29dbab81a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:15.798086 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48245e-b5b4-4e40-8723-76a29dbab81a None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:15.798956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48245e-b5b4-4e40-8723-76a29dbab81a None admin] RBAC: Adding query filter params (name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:15.799433 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48245e-b5b4-4e40-8723-76a29dbab81a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:15.802799 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48245e-b5b4-4e40-8723-76a29dbab81a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:15.820861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 245/485] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:15.821943 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8853731-61f6-4346-a20d-c4b2214f2549 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:15.822232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8853731-61f6-4346-a20d-c4b2214f2549 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:15.822505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8853731-61f6-4346-a20d-c4b2214f2549 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:15.828590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 246/486] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:15.828590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8853731-61f6-4346-a20d-c4b2214f2549 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:15.851324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 241/487] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:15.851324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20449478-88d9-4824-9a93-2d8fd0f2f688 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:15.851324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20449478-88d9-4824-9a93-2d8fd0f2f688 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:15.851324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20449478-88d9-4824-9a93-2d8fd0f2f688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:15.854059 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:15.880031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-20449478-88d9-4824-9a93-2d8fd0f2f688 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:15.882734 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-20449478-88d9-4824-9a93-2d8fd0f2f688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:16.018869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 242/488] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:16.026109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8853731-61f6-4346-a20d-c4b2214f2549 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:16.064972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46b4f0-fab1-45c2-85f2-76bc0e7e921a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:16.065938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46b4f0-fab1-45c2-85f2-76bc0e7e921a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:16.066411 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46b4f0-fab1-45c2-85f2-76bc0e7e921a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:16.066922 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46b4f0-fab1-45c2-85f2-76bc0e7e921a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:16.079006 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46b4f0-fab1-45c2-85f2-76bc0e7e921a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:16.080865 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46b4f0-fab1-45c2-85f2-76bc0e7e921a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-2f46b4f0-fab1-45c2-85f2-76bc0e7e921a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:16.101148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:16.102092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:16.102912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:16.102912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:16.102912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:16.104819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 247/489] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:16.110433 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20449478-88d9-4824-9a93-2d8fd0f2f688 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:16.142445 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:16.182004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46b4f0-fab1-45c2-85f2-76bc0e7e921a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:16.226072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20449478-88d9-4824-9a93-2d8fd0f2f688 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:16.269777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44af25-6f21-4f0d-bee3-48dfef1ed55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:16.270416 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44af25-6f21-4f0d-bee3-48dfef1ed55a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:16.270699 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44af25-6f21-4f0d-bee3-48dfef1ed55a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:16.271086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44af25-6f21-4f0d-bee3-48dfef1ed55a None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:16.271743 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44af25-6f21-4f0d-bee3-48dfef1ed55a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:16.272050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44af25-6f21-4f0d-bee3-48dfef1ed55a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:16.276819 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44af25-6f21-4f0d-bee3-48dfef1ed55a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:16.288674 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 243/490] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:16.290773 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44af25-6f21-4f0d-bee3-48dfef1ed55a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:16.306774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:16.375784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46b4f0-fab1-45c2-85f2-76bc0e7e921a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:16.422511 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:16.438441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b3708-e76c-42f8-ad20-ccaf112e7f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:16.439122 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b3708-e76c-42f8-ad20-ccaf112e7f8f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:16.439413 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b3708-e76c-42f8-ad20-ccaf112e7f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:16.439686 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b3708-e76c-42f8-ad20-ccaf112e7f8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:16.447879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44af25-6f21-4f0d-bee3-48dfef1ed55a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:16.451319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b3708-e76c-42f8-ad20-ccaf112e7f8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-012b3708-e76c-42f8-ad20-ccaf112e7f8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:16.456101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:16.457200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:16.463075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:16.463075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:16.463075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:16.463075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:16.463075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:16.463075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:16.463075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:16.463075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:16.463075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:16.463075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:16.463075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:16.463075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:16.463075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 248/491] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:01:16.465279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b3708-e76c-42f8-ad20-ccaf112e7f8f None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:16.500717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb284ac-1892-4aaf-8d61-7d03b7be9ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:16.501813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb284ac-1892-4aaf-8d61-7d03b7be9ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:16.502386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb284ac-1892-4aaf-8d61-7d03b7be9ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:16.502846 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb284ac-1892-4aaf-8d61-7d03b7be9ba8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:16.511768 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb284ac-1892-4aaf-8d61-7d03b7be9ba8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:16.513616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb284ac-1892-4aaf-8d61-7d03b7be9ba8 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-beb284ac-1892-4aaf-8d61-7d03b7be9ba8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:16.528452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:16.529539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:16.530478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:16.530478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:16.530478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:16.530478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:16.530478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:16.530478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:16.530478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:16.530478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:16.530478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:01:16.530478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:16.532894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 244/492] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 12 12:01:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:16.534050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb284ac-1892-4aaf-8d61-7d03b7be9ba8 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:16.594705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b3708-e76c-42f8-ad20-ccaf112e7f8f None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:16.621133 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:16.638180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605845d5-d786-4591-aa24-bb7decbeee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:16.639094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605845d5-d786-4591-aa24-bb7decbeee8f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:16.639565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605845d5-d786-4591-aa24-bb7decbeee8f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:16.640025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605845d5-d786-4591-aa24-bb7decbeee8f None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:16.641177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605845d5-d786-4591-aa24-bb7decbeee8f None admin] RBAC: Adding query filter params (name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:16.641811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605845d5-d786-4591-aa24-bb7decbeee8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:16.643429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605845d5-d786-4591-aa24-bb7decbeee8f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:16.652108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb284ac-1892-4aaf-8d61-7d03b7be9ba8 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:16.653422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 249/493] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:16.654570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605845d5-d786-4591-aa24-bb7decbeee8f None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:16.704950 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e370b2-1886-4991-b55a-a47d45776bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:16.705577 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e370b2-1886-4991-b55a-a47d45776bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:16.705853 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e370b2-1886-4991-b55a-a47d45776bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:16.706120 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e370b2-1886-4991-b55a-a47d45776bf9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:16.717687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e370b2-1886-4991-b55a-a47d45776bf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-c5e370b2-1886-4991-b55a-a47d45776bf9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:16.723345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:16.724218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:16.725717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:16.725717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:16.725717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:16.725717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:16.725717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:16.725717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:16.725717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:16.725717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:16.725717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:16.725717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:16.725717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:16.725717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:16.725717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 245/494] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:16.729158 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e370b2-1886-4991-b55a-a47d45776bf9 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:16.732882 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:16.772667 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605845d5-d786-4591-aa24-bb7decbeee8f None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:16.811665 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208c1eb-653d-4e2d-bf04-badfa399b32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:16.812162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208c1eb-653d-4e2d-bf04-badfa399b32c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:16.812446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208c1eb-653d-4e2d-bf04-badfa399b32c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:16.812765 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208c1eb-653d-4e2d-bf04-badfa399b32c None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:16.813505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208c1eb-653d-4e2d-bf04-badfa399b32c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:16.813817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208c1eb-653d-4e2d-bf04-badfa399b32c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:16.815723 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208c1eb-653d-4e2d-bf04-badfa399b32c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:16.850097 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e370b2-1886-4991-b55a-a47d45776bf9 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:16.859410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 250/495] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 12 12:01:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:16.859410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13cee2-9193-424d-a734-90b2c1527a6f None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:16.859410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13cee2-9193-424d-a734-90b2c1527a6f None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:16.859410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13cee2-9193-424d-a734-90b2c1527a6f None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:16.859410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 251/496] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:16.859410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13cee2-9193-424d-a734-90b2c1527a6f None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:16.890267 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e526de5-f35b-4602-8c54-d63657340d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:16.890875 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e526de5-f35b-4602-8c54-d63657340d40 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:16.891153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e526de5-f35b-4602-8c54-d63657340d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:16.891426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e526de5-f35b-4602-8c54-d63657340d40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:16.902755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e526de5-f35b-4602-8c54-d63657340d40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:16.904205 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e526de5-f35b-4602-8c54-d63657340d40 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:16.906527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-4e526de5-f35b-4602-8c54-d63657340d40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:16.919733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:16.932622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:16.933492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:16.933492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:16.933492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:16.933492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 246/497] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:16.933492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e526de5-f35b-4602-8c54-d63657340d40 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:16.978826 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13cee2-9193-424d-a734-90b2c1527a6f None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:16.983638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:17.016756 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa3353-c846-4c3a-97de-20df3fa2c948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:17.017413 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa3353-c846-4c3a-97de-20df3fa2c948 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:17.017808 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa3353-c846-4c3a-97de-20df3fa2c948 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:17.018078 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa3353-c846-4c3a-97de-20df3fa2c948 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:17.018740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa3353-c846-4c3a-97de-20df3fa2c948 None admin] RBAC: Adding query filter params (name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:17.019041 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa3353-c846-4c3a-97de-20df3fa2c948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:17.020469 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa3353-c846-4c3a-97de-20df3fa2c948 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:17.033500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 252/498] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:17.034082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e526de5-f35b-4602-8c54-d63657340d40 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:17.034424 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd0610-a652-46fe-9217-5a02e8b48e67 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:17.034424 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd0610-a652-46fe-9217-5a02e8b48e67 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:17.034424 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd0610-a652-46fe-9217-5a02e8b48e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:17.049616 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-6fcd0610-a652-46fe-9217-5a02e8b48e67 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:17.050804 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-6fcd0610-a652-46fe-9217-5a02e8b48e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:17.068287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd609a0c-657f-4e90-8c3c-b0c3a22bf3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:17.068949 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd609a0c-657f-4e90-8c3c-b0c3a22bf3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:17.069266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd609a0c-657f-4e90-8c3c-b0c3a22bf3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:17.069543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd609a0c-657f-4e90-8c3c-b0c3a22bf3d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:17.121450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 253/499] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:17.122293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd0610-a652-46fe-9217-5a02e8b48e67 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:17.154149 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:17.154337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd609a0c-657f-4e90-8c3c-b0c3a22bf3d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:17.155644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd609a0c-657f-4e90-8c3c-b0c3a22bf3d5 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:17.212553 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:17.228342 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd0610-a652-46fe-9217-5a02e8b48e67 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:17.261105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 247/500] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:17.262218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd609a0c-657f-4e90-8c3c-b0c3a22bf3d5 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:17.270998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfcc7d-4b7c-412f-8c27-9660b7c8798c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:17.271611 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfcc7d-4b7c-412f-8c27-9660b7c8798c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:17.271893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfcc7d-4b7c-412f-8c27-9660b7c8798c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:17.272162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfcc7d-4b7c-412f-8c27-9660b7c8798c None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:17.273219 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfcc7d-4b7c-412f-8c27-9660b7c8798c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:17.273526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfcc7d-4b7c-412f-8c27-9660b7c8798c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:17.275319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfcc7d-4b7c-412f-8c27-9660b7c8798c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:17.291117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 254/501] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:17.291117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8f8ab-a60e-48f4-949b-75186e60be03 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:17.291117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8f8ab-a60e-48f4-949b-75186e60be03 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:17.291117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8f8ab-a60e-48f4-949b-75186e60be03 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:17.291117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 255/502] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:17.294377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8f8ab-a60e-48f4-949b-75186e60be03 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:17.375777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd609a0c-657f-4e90-8c3c-b0c3a22bf3d5 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:17.391152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:17.401242 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8f8ab-a60e-48f4-949b-75186e60be03 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:17.410503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecb0f2-7d5c-41eb-9a34-33bf6a8c59a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:17.411318 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecb0f2-7d5c-41eb-9a34-33bf6a8c59a1 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:17.411611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecb0f2-7d5c-41eb-9a34-33bf6a8c59a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:17.411881 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecb0f2-7d5c-41eb-9a34-33bf6a8c59a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:17.422229 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adecb0f2-7d5c-41eb-9a34-33bf6a8c59a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:17.423718 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adecb0f2-7d5c-41eb-9a34-33bf6a8c59a1 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:17.434148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa6dbd-3291-4221-aeaa-9d4a9096a915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:17.434781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa6dbd-3291-4221-aeaa-9d4a9096a915 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:17.435073 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa6dbd-3291-4221-aeaa-9d4a9096a915 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:17.435541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa6dbd-3291-4221-aeaa-9d4a9096a915 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-adecb0f2-7d5c-41eb-9a34-33bf6a8c59a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:17.438806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:17.439661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:17.440517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:17.440517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:17.440517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:17.440517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 248/503] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:17.440765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecb0f2-7d5c-41eb-9a34-33bf6a8c59a1 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:17.447448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa6dbd-3291-4221-aeaa-9d4a9096a915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:17.448965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa6dbd-3291-4221-aeaa-9d4a9096a915 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:17.450022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-18aa6dbd-3291-4221-aeaa-9d4a9096a915 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:17.463468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:17.464418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:17.465708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:17.465708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:17.465708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:17.465708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 256/504] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:17.465997 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4348e-bdac-4891-9b90-a52267ad6670 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:17.466075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4348e-bdac-4891-9b90-a52267ad6670 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:17.466335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4348e-bdac-4891-9b90-a52267ad6670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:17.485842 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-cac4348e-bdac-4891-9b90-a52267ad6670 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:17.487036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-cac4348e-bdac-4891-9b90-a52267ad6670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:17.543332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecb0f2-7d5c-41eb-9a34-33bf6a8c59a1 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:17.554212 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 257/505] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:17.555298 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4348e-bdac-4891-9b90-a52267ad6670 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:17.581631 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea279a-9230-4bd6-85d9-c8efe26693ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:17.582252 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea279a-9230-4bd6-85d9-c8efe26693ae None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:17.582606 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea279a-9230-4bd6-85d9-c8efe26693ae None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:17.582921 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea279a-9230-4bd6-85d9-c8efe26693ae None admin] PATH_INFO: `/v3/users/glance` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:17.590688 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea279a-9230-4bd6-85d9-c8efe26693ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:17.592148 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea279a-9230-4bd6-85d9-c8efe26693ae None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-9bea279a-9230-4bd6-85d9-c8efe26693ae None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:17.604135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:17.605040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:17.605893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:17.605893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:17.605893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:17.605893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:17.605893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:17.605893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:17.605893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:17.605893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:17.605893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 12 12:01:17.605893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:17.605893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 249/506] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:17.606429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea279a-9230-4bd6-85d9-c8efe26693ae None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:17.646827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:17.672576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4348e-bdac-4891-9b90-a52267ad6670 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:17.675952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:17.709387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a83c1a-0e5f-4839-9fa6-4e1200a2c7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:17.710023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a83c1a-0e5f-4839-9fa6-4e1200a2c7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:17.710306 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a83c1a-0e5f-4839-9fa6-4e1200a2c7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:17.710579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a83c1a-0e5f-4839-9fa6-4e1200a2c7a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:17.714400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea279a-9230-4bd6-85d9-c8efe26693ae None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:17.721474 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a83c1a-0e5f-4839-9fa6-4e1200a2c7a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-d6a83c1a-0e5f-4839-9fa6-4e1200a2c7a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:17.726669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:17.728344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:17.728344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:17.728344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:17.728344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:17.728344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:17.728344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:17.728344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:17.728344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:17.728344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:17.728344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:17.728344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:17.728344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:17.728344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 258/507] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:17.729523 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a83c1a-0e5f-4839-9fa6-4e1200a2c7a8 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:17.757182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e6b16-2523-47bd-a461-a1d94ff318c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:17.757924 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e6b16-2523-47bd-a461-a1d94ff318c4 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:17.758306 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e6b16-2523-47bd-a461-a1d94ff318c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:17.758698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e6b16-2523-47bd-a461-a1d94ff318c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:17.759443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e6b16-2523-47bd-a461-a1d94ff318c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:17.759850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e6b16-2523-47bd-a461-a1d94ff318c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:17.761358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e6b16-2523-47bd-a461-a1d94ff318c4 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:17.778203 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 250/508] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:17.778203 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e6b16-2523-47bd-a461-a1d94ff318c4 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:17.844185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a83c1a-0e5f-4839-9fa6-4e1200a2c7a8 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:17.877236 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535ebd7-19c1-4223-8f11-2b58e8db4674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:17.877833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535ebd7-19c1-4223-8f11-2b58e8db4674 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:17.878111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535ebd7-19c1-4223-8f11-2b58e8db4674 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:17.878392 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535ebd7-19c1-4223-8f11-2b58e8db4674 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:17.879029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535ebd7-19c1-4223-8f11-2b58e8db4674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:17.879910 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535ebd7-19c1-4223-8f11-2b58e8db4674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:17.881526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535ebd7-19c1-4223-8f11-2b58e8db4674 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:17.882941 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:17.885042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e6b16-2523-47bd-a461-a1d94ff318c4 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:17.893466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 259/509] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:17.894271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535ebd7-19c1-4223-8f11-2b58e8db4674 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:17.911566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:17.919574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd87cd9-4135-42d2-9e9d-0056cb187742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:17.920164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd87cd9-4135-42d2-9e9d-0056cb187742 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:17.920443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd87cd9-4135-42d2-9e9d-0056cb187742 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:17.920744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd87cd9-4135-42d2-9e9d-0056cb187742 None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:17.921470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd87cd9-4135-42d2-9e9d-0056cb187742 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:17.921770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd87cd9-4135-42d2-9e9d-0056cb187742 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:17.923201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd87cd9-4135-42d2-9e9d-0056cb187742 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:17.960199 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 251/510] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Aug 12 12:01:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:17.961051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd87cd9-4135-42d2-9e9d-0056cb187742 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:17.994354 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535ebd7-19c1-4223-8f11-2b58e8db4674 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:18.028694 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04f953-16d8-4f7b-aa36-102f5cf4c82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:18.029300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04f953-16d8-4f7b-aa36-102f5cf4c82c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:18.029581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04f953-16d8-4f7b-aa36-102f5cf4c82c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:18.029857 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04f953-16d8-4f7b-aa36-102f5cf4c82c None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:18.030527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04f953-16d8-4f7b-aa36-102f5cf4c82c None admin] RBAC: Adding query filter params (name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:18.030829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04f953-16d8-4f7b-aa36-102f5cf4c82c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:18.032226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04f953-16d8-4f7b-aa36-102f5cf4c82c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:18.040171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 260/511] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:18.040998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04f953-16d8-4f7b-aa36-102f5cf4c82c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:18.061268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd87cd9-4135-42d2-9e9d-0056cb187742 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:18.095079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b8c6e-955b-45cc-8f75-0748ad1b7e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:18.095698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b8c6e-955b-45cc-8f75-0748ad1b7e4a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:18.095985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b8c6e-955b-45cc-8f75-0748ad1b7e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:18.096184 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:18.096295 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b8c6e-955b-45cc-8f75-0748ad1b7e4a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:18.103405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b8c6e-955b-45cc-8f75-0748ad1b7e4a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:18.104893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b8c6e-955b-45cc-8f75-0748ad1b7e4a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-d34b8c6e-955b-45cc-8f75-0748ad1b7e4a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:18.116787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:18.123676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:18.124586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:18.124586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:18.124586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:18.124586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:18.124586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:18.124586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:18.124586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:18.124586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:18.124586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 12:01:18.124586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:18.124586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 252/512] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:18.124586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b8c6e-955b-45cc-8f75-0748ad1b7e4a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:18.138353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04f953-16d8-4f7b-aa36-102f5cf4c82c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:18.164383 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:18.172365 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81314779-b293-4967-a078-5ddfebffed76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:18.173110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81314779-b293-4967-a078-5ddfebffed76 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:18.173299 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81314779-b293-4967-a078-5ddfebffed76 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:18.173570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81314779-b293-4967-a078-5ddfebffed76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:18.184934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81314779-b293-4967-a078-5ddfebffed76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:18.186384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81314779-b293-4967-a078-5ddfebffed76 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-81314779-b293-4967-a078-5ddfebffed76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:18.201396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:18.202285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:18.203106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:18.203106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:18.203106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:18.203106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 261/513] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:18.203318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81314779-b293-4967-a078-5ddfebffed76 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:18.221960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b8c6e-955b-45cc-8f75-0748ad1b7e4a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:18.254694 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089344a9-3ea8-4be7-9a01-861dd8e4eddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:18.255319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089344a9-3ea8-4be7-9a01-861dd8e4eddc None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:18.255605 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089344a9-3ea8-4be7-9a01-861dd8e4eddc None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:18.255881 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089344a9-3ea8-4be7-9a01-861dd8e4eddc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:18.262597 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089344a9-3ea8-4be7-9a01-861dd8e4eddc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:18.264054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089344a9-3ea8-4be7-9a01-861dd8e4eddc None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:18.276413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-089344a9-3ea8-4be7-9a01-861dd8e4eddc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:18.276413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:18.276413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:18.276413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:18.276413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:18.276413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:18.276413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:18.276413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:18.276413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:18.276413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:18.276413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:18.276413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:18.276413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:18.276413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:18.276413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:18.277278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:18.278390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:18.278390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:18.278390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:18.278390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:18.278390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:18.278390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:18.278390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:18.278390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:18.278390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 253/514] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:18.279141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089344a9-3ea8-4be7-9a01-861dd8e4eddc None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:18.306001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81314779-b293-4967-a078-5ddfebffed76 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:18.314812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:18.339135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72159471-6b13-46a8-9fa1-edcfbb7d302a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:18.339734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72159471-6b13-46a8-9fa1-edcfbb7d302a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:18.340016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72159471-6b13-46a8-9fa1-edcfbb7d302a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:18.340292 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72159471-6b13-46a8-9fa1-edcfbb7d302a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:18.378555 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089344a9-3ea8-4be7-9a01-861dd8e4eddc None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:18.381034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:18.406528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72159471-6b13-46a8-9fa1-edcfbb7d302a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:18.407998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72159471-6b13-46a8-9fa1-edcfbb7d302a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:18.412114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de28cd-1eae-40a2-9323-9469f0a7d6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:18.412900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de28cd-1eae-40a2-9323-9469f0a7d6ec None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:18.413482 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de28cd-1eae-40a2-9323-9469f0a7d6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:18.413755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de28cd-1eae-40a2-9323-9469f0a7d6ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:18.424040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de28cd-1eae-40a2-9323-9469f0a7d6ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:18.425504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de28cd-1eae-40a2-9323-9469f0a7d6ec None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-19de28cd-1eae-40a2-9323-9469f0a7d6ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:18.439332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:18.440233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:18.441110 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:18.441110 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:18.441110 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:18.445760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 254/515] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:18.446585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de28cd-1eae-40a2-9323-9469f0a7d6ec None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:18.506286 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 262/516] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:18.507121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72159471-6b13-46a8-9fa1-edcfbb7d302a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:18.516682 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:18.547932 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de28cd-1eae-40a2-9323-9469f0a7d6ec None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:18.581153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d622c0-1ad8-4494-90bb-20c11815b4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:18.581745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d622c0-1ad8-4494-90bb-20c11815b4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:18.582025 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d622c0-1ad8-4494-90bb-20c11815b4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:18.582297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d622c0-1ad8-4494-90bb-20c11815b4d0 None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:18.583154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d622c0-1ad8-4494-90bb-20c11815b4d0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:18.583450 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d622c0-1ad8-4494-90bb-20c11815b4d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:18.584883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d622c0-1ad8-4494-90bb-20c11815b4d0 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:18.602028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 255/517] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Aug 12 12:01:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:18.602822 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d622c0-1ad8-4494-90bb-20c11815b4d0 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:18.609496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72159471-6b13-46a8-9fa1-edcfbb7d302a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:18.617657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:18.648327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2517530-0e77-4103-a92a-9fd3c40e8058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:18.648965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2517530-0e77-4103-a92a-9fd3c40e8058 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:18.649249 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2517530-0e77-4103-a92a-9fd3c40e8058 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:18.649523 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2517530-0e77-4103-a92a-9fd3c40e8058 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:18.650159 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2517530-0e77-4103-a92a-9fd3c40e8058 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:18.650462 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2517530-0e77-4103-a92a-9fd3c40e8058 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:18.651834 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2517530-0e77-4103-a92a-9fd3c40e8058 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:18.663213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 263/518] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:18.663998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2517530-0e77-4103-a92a-9fd3c40e8058 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:18.702106 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d622c0-1ad8-4494-90bb-20c11815b4d0 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:18.729972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:18.736009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38671a6-2382-4606-b550-c9fec995fb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:18.736632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38671a6-2382-4606-b550-c9fec995fb6d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:18.737075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38671a6-2382-4606-b550-c9fec995fb6d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:18.737369 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38671a6-2382-4606-b550-c9fec995fb6d None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:18.738052 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38671a6-2382-4606-b550-c9fec995fb6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:18.738363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38671a6-2382-4606-b550-c9fec995fb6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:18.740474 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38671a6-2382-4606-b550-c9fec995fb6d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:18.758252 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 256/519] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:18.759074 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38671a6-2382-4606-b550-c9fec995fb6d None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:18.763941 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2517530-0e77-4103-a92a-9fd3c40e8058 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:18.800412 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fd57a-2777-48ba-a1ac-4b1233d2a1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:18.801031 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fd57a-2777-48ba-a1ac-4b1233d2a1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:18.801482 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fd57a-2777-48ba-a1ac-4b1233d2a1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:18.801871 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fd57a-2777-48ba-a1ac-4b1233d2a1a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:18.802621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fd57a-2777-48ba-a1ac-4b1233d2a1a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:18.802922 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fd57a-2777-48ba-a1ac-4b1233d2a1a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:18.804297 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fd57a-2777-48ba-a1ac-4b1233d2a1a2 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:18.818999 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 264/520] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:18.820002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fd57a-2777-48ba-a1ac-4b1233d2a1a2 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:18.835915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:18.858929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38671a6-2382-4606-b550-c9fec995fb6d None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:18.892247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848d03a-bf5f-4451-8993-252c454b813b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:18.892898 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848d03a-bf5f-4451-8993-252c454b813b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:18.893180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848d03a-bf5f-4451-8993-252c454b813b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:18.893452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848d03a-bf5f-4451-8993-252c454b813b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:18.903318 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848d03a-bf5f-4451-8993-252c454b813b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:18.904805 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848d03a-bf5f-4451-8993-252c454b813b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:18.916931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fd57a-2777-48ba-a1ac-4b1233d2a1a2 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-0848d03a-bf5f-4451-8993-252c454b813b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:18.918808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:18.919643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:18.920480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:18.920480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:18.920480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:18.920480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 257/521] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:18.920849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848d03a-bf5f-4451-8993-252c454b813b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:18.945605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:18.953961 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fd8b1-c96e-45c3-82f1-fef7845df3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:18.954095 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fd8b1-c96e-45c3-82f1-fef7845df3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:18.954429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fd8b1-c96e-45c3-82f1-fef7845df3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:18.954695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fd8b1-c96e-45c3-82f1-fef7845df3b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:19.022726 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fd8b1-c96e-45c3-82f1-fef7845df3b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:19.024225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fd8b1-c96e-45c3-82f1-fef7845df3b6 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:19.024776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848d03a-bf5f-4451-8993-252c454b813b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:19.049912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:19.059069 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdf5c0-f4c9-4e14-9e6b-32744ec53d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:19.059668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdf5c0-f4c9-4e14-9e6b-32744ec53d7d None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:19.059953 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdf5c0-f4c9-4e14-9e6b-32744ec53d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:19.060229 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdf5c0-f4c9-4e14-9e6b-32744ec53d7d None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:19.060905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cdf5c0-f4c9-4e14-9e6b-32744ec53d7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:19.062333 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cdf5c0-f4c9-4e14-9e6b-32744ec53d7d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:19.121965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 265/522] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:19.122799 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fd8b1-c96e-45c3-82f1-fef7845df3b6 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:19.143787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 258/523] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:01:18 2022] POST /identity/v3/users => generated 247 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:01:19.145510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdf5c0-f4c9-4e14-9e6b-32744ec53d7d None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:19.157129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:19.236633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fd8b1-c96e-45c3-82f1-fef7845df3b6 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:19.269539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdf5c0-f4c9-4e14-9e6b-32744ec53d7d None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:19.274814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f360432-9ed5-4ffc-a5c5-493328892b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:19.275535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f360432-9ed5-4ffc-a5c5-493328892b43 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:19.275819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f360432-9ed5-4ffc-a5c5-493328892b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:19.276088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f360432-9ed5-4ffc-a5c5-493328892b43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:19.276853 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f360432-9ed5-4ffc-a5c5-493328892b43 None admin] RBAC: Adding query filter params (role.id=1416a05bf5ba4de88889c54ff5c58da2, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=687d0375abd44d2481f988c6df90fad1) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:19.277161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f360432-9ed5-4ffc-a5c5-493328892b43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:19.278688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f360432-9ed5-4ffc-a5c5-493328892b43 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:19.284723 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 266/524] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:19 2022] GET /identity/v3/role_assignments?user.id=687d0375abd44d2481f988c6df90fad1&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=1416a05bf5ba4de88889c54ff5c58da2 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:19.286275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f360432-9ed5-4ffc-a5c5-493328892b43 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:19.314811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f3415-e77e-4e2d-914f-6b166e078970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:19.315462 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f3415-e77e-4e2d-914f-6b166e078970 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:19.315746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f3415-e77e-4e2d-914f-6b166e078970 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:19.316019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f3415-e77e-4e2d-914f-6b166e078970 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:19.325393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f3415-e77e-4e2d-914f-6b166e078970 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:19.326872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f3415-e77e-4e2d-914f-6b166e078970 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-9d6f3415-e77e-4e2d-914f-6b166e078970 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:19.341439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:19.342443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:19.343297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:19.343297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:19.343297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:19.343297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:19.343297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:19.343297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:19.343297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:19.343297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:19.343297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 12 12:01:19.343297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:19.343297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 259/525] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:19.343297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f3415-e77e-4e2d-914f-6b166e078970 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:19.406761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:19.417890 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:19.429793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f360432-9ed5-4ffc-a5c5-493328892b43 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:19.471962 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f3415-e77e-4e2d-914f-6b166e078970 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:19.472274 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0ec67-d330-4251-b782-fd0e5402de2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:19.473639 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0ec67-d330-4251-b782-fd0e5402de2d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:19.474011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0ec67-d330-4251-b782-fd0e5402de2d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:19.474287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0ec67-d330-4251-b782-fd0e5402de2d None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:19.474938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0ec67-d330-4251-b782-fd0e5402de2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:19.475245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0ec67-d330-4251-b782-fd0e5402de2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:19.476651 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0ec67-d330-4251-b782-fd0e5402de2d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:19.494023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 267/526] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:19.494023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0ec67-d330-4251-b782-fd0e5402de2d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:19.522463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f7e0a-dc33-44dc-8ef5-6985017e434e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:19.523102 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f7e0a-dc33-44dc-8ef5-6985017e434e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:19.523386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f7e0a-dc33-44dc-8ef5-6985017e434e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:19.523716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f7e0a-dc33-44dc-8ef5-6985017e434e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:19.536001 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f7e0a-dc33-44dc-8ef5-6985017e434e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:19.537778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f7e0a-dc33-44dc-8ef5-6985017e434e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-ec0f7e0a-dc33-44dc-8ef5-6985017e434e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:19.553132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:19.554151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:19.555235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:19.555235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:19.555235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:19.555235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 260/527] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:19.555869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f7e0a-dc33-44dc-8ef5-6985017e434e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:19.612858 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0ec67-d330-4251-b782-fd0e5402de2d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:19.664265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd1ba6-bbe0-48dc-a51d-d07e7914b143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:19.664896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd1ba6-bbe0-48dc-a51d-d07e7914b143 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:19.665282 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd1ba6-bbe0-48dc-a51d-d07e7914b143 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:19.665560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd1ba6-bbe0-48dc-a51d-d07e7914b143 None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:19.666402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd1ba6-bbe0-48dc-a51d-d07e7914b143 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:19.666707 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd1ba6-bbe0-48dc-a51d-d07e7914b143 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:19.668399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:19.669161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd1ba6-bbe0-48dc-a51d-d07e7914b143 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:19.675393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f7e0a-dc33-44dc-8ef5-6985017e434e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:19.688411 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 268/528] 10.222.0.27 () {56 vars in 1139 bytes} [Fri Aug 12 12:01:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:19.689339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd1ba6-bbe0-48dc-a51d-d07e7914b143 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:19.702210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:19.714040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317943ac-6c5e-4087-9981-10449f6c9260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:19.714720 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317943ac-6c5e-4087-9981-10449f6c9260 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:19.715004 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317943ac-6c5e-4087-9981-10449f6c9260 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:19.715277 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317943ac-6c5e-4087-9981-10449f6c9260 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:19.722815 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317943ac-6c5e-4087-9981-10449f6c9260 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:19.724566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317943ac-6c5e-4087-9981-10449f6c9260 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:19.739686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-317943ac-6c5e-4087-9981-10449f6c9260 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:19.739686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:19.739686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:19.739686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:19.739686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:19.739686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:19.739686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:19.739686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:19.739686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:19.739686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:19.739686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:19.739686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:19.739686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:19.739686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:19.739686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:19.741731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:19.743179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:19.743179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:19.743179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:19.743179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:19.743179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:19.743179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:19.743179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:19.743179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:19.748129 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 261/529] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:19.748129 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317943ac-6c5e-4087-9981-10449f6c9260 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:19.795545 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd1ba6-bbe0-48dc-a51d-d07e7914b143 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:19.832812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab4c90-4c25-4a32-a9f3-7e68b8d8cade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:19.833413 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab4c90-4c25-4a32-a9f3-7e68b8d8cade None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:19.833691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab4c90-4c25-4a32-a9f3-7e68b8d8cade None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:19.833962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab4c90-4c25-4a32-a9f3-7e68b8d8cade None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:19.834601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab4c90-4c25-4a32-a9f3-7e68b8d8cade None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:19.834903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab4c90-4c25-4a32-a9f3-7e68b8d8cade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:19.836277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab4c90-4c25-4a32-a9f3-7e68b8d8cade None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:19.848340 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 269/530] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:19.849232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab4c90-4c25-4a32-a9f3-7e68b8d8cade None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:19.867524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317943ac-6c5e-4087-9981-10449f6c9260 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:19.904060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792bfa4-7e74-400b-b433-52787948802a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:19.905017 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792bfa4-7e74-400b-b433-52787948802a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:19.905479 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792bfa4-7e74-400b-b433-52787948802a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:19.905924 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792bfa4-7e74-400b-b433-52787948802a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:19.922863 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:19.934219 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:19.954718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab4c90-4c25-4a32-a9f3-7e68b8d8cade None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:19.981158 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792bfa4-7e74-400b-b433-52787948802a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:19.982897 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792bfa4-7e74-400b-b433-52787948802a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:19.990089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df65d3-c61f-4c1a-9d0b-20d071203289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:19.990695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df65d3-c61f-4c1a-9d0b-20d071203289 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:19.990976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df65d3-c61f-4c1a-9d0b-20d071203289 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:19.991258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df65d3-c61f-4c1a-9d0b-20d071203289 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:19.992143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df65d3-c61f-4c1a-9d0b-20d071203289 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:19.992251 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df65d3-c61f-4c1a-9d0b-20d071203289 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:19.993838 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df65d3-c61f-4c1a-9d0b-20d071203289 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:20.018908 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 270/531] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:20.018908 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df65d3-c61f-4c1a-9d0b-20d071203289 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:20.096437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 262/532] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:20.098533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792bfa4-7e74-400b-b433-52787948802a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:20.121943 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df65d3-c61f-4c1a-9d0b-20d071203289 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:20.150835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:20.157046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee97243-9a0c-49f2-aa11-74d405f4a8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:20.157670 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee97243-9a0c-49f2-aa11-74d405f4a8ea None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:20.157949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee97243-9a0c-49f2-aa11-74d405f4a8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:20.158222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee97243-9a0c-49f2-aa11-74d405f4a8ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:20.158875 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:20.165741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee97243-9a0c-49f2-aa11-74d405f4a8ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:20.167330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee97243-9a0c-49f2-aa11-74d405f4a8ea None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:20.179764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-6ee97243-9a0c-49f2-aa11-74d405f4a8ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:20.179764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:20.179764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:20.179764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:20.179764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:20.179764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:20.179764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:20.179764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:20.179764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:20.179764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:20.179764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:20.179764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:20.179764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:20.179764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:20.179764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:20.180917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:20.182054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:20.182054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:20.182054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:20.182054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:20.182054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:20.182054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:20.182054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:20.182054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:20.182054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 271/533] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:20.182705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee97243-9a0c-49f2-aa11-74d405f4a8ea None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:20.210742 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792bfa4-7e74-400b-b433-52787948802a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:20.248874 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5659c-429d-428a-a6b6-419ff14f637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:20.249834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5659c-429d-428a-a6b6-419ff14f637f None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:20.250296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5659c-429d-428a-a6b6-419ff14f637f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:20.250762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5659c-429d-428a-a6b6-419ff14f637f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:20.251667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5659c-429d-428a-a6b6-419ff14f637f None admin] RBAC: Adding query filter params (role.id=1416a05bf5ba4de88889c54ff5c58da2, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=ffe9f3fec7104a289e05e88e2d751458) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:20.252222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5659c-429d-428a-a6b6-419ff14f637f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:20.253872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5659c-429d-428a-a6b6-419ff14f637f None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:20.259951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 263/534] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:20 2022] GET /identity/v3/role_assignments?user.id=ffe9f3fec7104a289e05e88e2d751458&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=1416a05bf5ba4de88889c54ff5c58da2 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:20.261065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5659c-429d-428a-a6b6-419ff14f637f None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:20.289927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee97243-9a0c-49f2-aa11-74d405f4a8ea None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:20.327974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e443d1f-abca-464b-8a11-b65b761d77af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:20.328603 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e443d1f-abca-464b-8a11-b65b761d77af None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:20.328915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e443d1f-abca-464b-8a11-b65b761d77af None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:20.329191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e443d1f-abca-464b-8a11-b65b761d77af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:20.342084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e443d1f-abca-464b-8a11-b65b761d77af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:20.343567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e443d1f-abca-464b-8a11-b65b761d77af None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-1e443d1f-abca-464b-8a11-b65b761d77af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:20.359988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:20.360977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:20.362250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:20.362250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:20.362250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:20.362250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 272/535] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:20.367816 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e443d1f-abca-464b-8a11-b65b761d77af None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:20.388211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5659c-429d-428a-a6b6-419ff14f637f None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:20.409005 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:20.432282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57c21f-a6b5-4322-a9a5-3581bdf1ade8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:20.434160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57c21f-a6b5-4322-a9a5-3581bdf1ade8 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:20.434702 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57c21f-a6b5-4322-a9a5-3581bdf1ade8 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:20.435188 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57c21f-a6b5-4322-a9a5-3581bdf1ade8 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:20.436096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57c21f-a6b5-4322-a9a5-3581bdf1ade8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:20.436604 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57c21f-a6b5-4322-a9a5-3581bdf1ade8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:20.438324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57c21f-a6b5-4322-a9a5-3581bdf1ade8 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:20.438653 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:20.454487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 264/536] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:20.460896 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57c21f-a6b5-4322-a9a5-3581bdf1ade8 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:20.555148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e443d1f-abca-464b-8a11-b65b761d77af None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:20.585939 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57c21f-a6b5-4322-a9a5-3581bdf1ade8 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:20.624872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b656f-e82b-4aeb-8eb4-a308f7c5a613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:20.626094 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b656f-e82b-4aeb-8eb4-a308f7c5a613 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:20.626575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b656f-e82b-4aeb-8eb4-a308f7c5a613 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:20.627148 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b656f-e82b-4aeb-8eb4-a308f7c5a613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:20.628379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b656f-e82b-4aeb-8eb4-a308f7c5a613 None admin] RBAC: Adding query filter params (role.id=1416a05bf5ba4de88889c54ff5c58da2, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=935ab3a263ae46fcb929029a147a1010) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:20.629066 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b656f-e82b-4aeb-8eb4-a308f7c5a613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:20.631625 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b656f-e82b-4aeb-8eb4-a308f7c5a613 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:20.638122 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 265/537] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:20 2022] GET /identity/v3/role_assignments?user.id=935ab3a263ae46fcb929029a147a1010&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=1416a05bf5ba4de88889c54ff5c58da2 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:20.666003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e493cd-abbe-45e2-a732-bab90ef359bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:20.666624 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e493cd-abbe-45e2-a732-bab90ef359bf None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:20.666906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e493cd-abbe-45e2-a732-bab90ef359bf None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:20.667178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e493cd-abbe-45e2-a732-bab90ef359bf None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:20.667831 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e493cd-abbe-45e2-a732-bab90ef359bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:20.668133 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e493cd-abbe-45e2-a732-bab90ef359bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:20.669620 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e493cd-abbe-45e2-a732-bab90ef359bf None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:20.695587 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 273/538] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:20.701402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b656f-e82b-4aeb-8eb4-a308f7c5a613 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:20.769411 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:20.840854 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b656f-e82b-4aeb-8eb4-a308f7c5a613 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:20.887737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e427df9-efa4-4114-a209-fc7499aca9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:20.888368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e427df9-efa4-4114-a209-fc7499aca9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:20.888672 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e427df9-efa4-4114-a209-fc7499aca9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:20.888940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e427df9-efa4-4114-a209-fc7499aca9e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:20.900100 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e427df9-efa4-4114-a209-fc7499aca9e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:20.901599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e427df9-efa4-4114-a209-fc7499aca9e3 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:20.914086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-9e427df9-efa4-4114-a209-fc7499aca9e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:20.914086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:20.914086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:20.914086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:20.914086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:20.914086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:20.914086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:20.914086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:20.914086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:20.914086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:20.914086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:20.914086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:20.914086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:20.914086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:20.914086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:20.914935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:20.916202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:20.916202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:20.916202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:20.916202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:20.916202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:20.916202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:20.916202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:20.916202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:20.916202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 266/539] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:20.921421 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e493cd-abbe-45e2-a732-bab90ef359bf None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:21.058106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e493cd-abbe-45e2-a732-bab90ef359bf None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:21.059717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:21.103170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9e6cc-fb66-498a-91b6-dd47f54566a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:21.103781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9e6cc-fb66-498a-91b6-dd47f54566a6 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:21.104062 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9e6cc-fb66-498a-91b6-dd47f54566a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:21.104336 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9e6cc-fb66-498a-91b6-dd47f54566a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:21.105054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9e6cc-fb66-498a-91b6-dd47f54566a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:21.105362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9e6cc-fb66-498a-91b6-dd47f54566a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:21.106787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9e6cc-fb66-498a-91b6-dd47f54566a6 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:21.123468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 274/540] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:21.129435 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e427df9-efa4-4114-a209-fc7499aca9e3 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:21.235424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e427df9-efa4-4114-a209-fc7499aca9e3 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:21.236940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:21.272268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc410c56-588c-465d-8e6a-a4ee4ea427d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:21.272898 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc410c56-588c-465d-8e6a-a4ee4ea427d9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:21.273181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc410c56-588c-465d-8e6a-a4ee4ea427d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:21.273454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc410c56-588c-465d-8e6a-a4ee4ea427d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:21.274185 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc410c56-588c-465d-8e6a-a4ee4ea427d9 None admin] RBAC: Adding query filter params (role.id=1416a05bf5ba4de88889c54ff5c58da2, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=745fad9c109a4280906eb0349849117d) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:21.274795 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc410c56-588c-465d-8e6a-a4ee4ea427d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:21.276740 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc410c56-588c-465d-8e6a-a4ee4ea427d9 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:21.282759 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 267/541] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:21 2022] GET /identity/v3/role_assignments?user.id=745fad9c109a4280906eb0349849117d&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=1416a05bf5ba4de88889c54ff5c58da2 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:21.405839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb85d8-83c3-4662-949a-cc05f07e99c9 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:21.406340 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb85d8-83c3-4662-949a-cc05f07e99c9 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:21.406440 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb85d8-83c3-4662-949a-cc05f07e99c9 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:21.408016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 275/542] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:21.417334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c9fb1-0f15-49aa-bb5d-c7c98e20e358 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:21.417636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c9fb1-0f15-49aa-bb5d-c7c98e20e358 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:21.417893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c9fb1-0f15-49aa-bb5d-c7c98e20e358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:21.442445 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-8f2c9fb1-0f15-49aa-bb5d-c7c98e20e358 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:21.443697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8f2c9fb1-0f15-49aa-bb5d-c7c98e20e358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:21.535776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 268/543] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:21.585645 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15338661-4230-4f20-9686-c5c5e3603fa8 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:21.585974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15338661-4230-4f20-9686-c5c5e3603fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:21.586198 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15338661-4230-4f20-9686-c5c5e3603fa8 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:21.587787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 276/544] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:21.592682 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b04ce7-a9b7-4050-bda3-2603a7c5f0d7 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:21.592978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b04ce7-a9b7-4050-bda3-2603a7c5f0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:21.593256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b04ce7-a9b7-4050-bda3-2603a7c5f0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:21.616705 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-05b04ce7-a9b7-4050-bda3-2603a7c5f0d7 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:21.617973 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-05b04ce7-a9b7-4050-bda3-2603a7c5f0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:21.706360 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 269/545] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:21.866153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15338661-4230-4f20-9686-c5c5e3603fa8 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:21.982323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b04ce7-a9b7-4050-bda3-2603a7c5f0d7 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:22.003790 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15338661-4230-4f20-9686-c5c5e3603fa8 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:22.017356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:22.075819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20084e-4c2e-46af-8dec-8d00d911423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:22.076870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20084e-4c2e-46af-8dec-8d00d911423f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:22.077353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20084e-4c2e-46af-8dec-8d00d911423f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:22.077822 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20084e-4c2e-46af-8dec-8d00d911423f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:22.101123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20084e-4c2e-46af-8dec-8d00d911423f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-7b20084e-4c2e-46af-8dec-8d00d911423f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:22.106896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:22.107830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:22.108598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:22.108598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:22.108598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:22.108598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:22.108598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:22.108598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:22.108598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:22.108598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:22.108598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:22.108598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:22.108598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:22.108598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:22.113905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 277/546] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:22.121331 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20084e-4c2e-46af-8dec-8d00d911423f None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:22.154800 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b04ce7-a9b7-4050-bda3-2603a7c5f0d7 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:22.174643 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:22.238293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338d7b6-8dd4-4113-999a-40e9db06fae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:22.239455 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338d7b6-8dd4-4113-999a-40e9db06fae6 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:22.239947 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338d7b6-8dd4-4113-999a-40e9db06fae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:22.240427 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338d7b6-8dd4-4113-999a-40e9db06fae6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:22.260594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338d7b6-8dd4-4113-999a-40e9db06fae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-8338d7b6-8dd4-4113-999a-40e9db06fae6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:22.266844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:22.267697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:22.268456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:22.268456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:22.268456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:22.268456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:22.268456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:22.268456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:22.268456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:22.268456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:22.268456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:22.268456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:22.268456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:22.268456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:22.276509 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 270/547] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:01:22.277999 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338d7b6-8dd4-4113-999a-40e9db06fae6 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:22.323117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20084e-4c2e-46af-8dec-8d00d911423f None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:22.387835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08433a96-c578-4065-9841-d6d59278cbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:22.388955 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08433a96-c578-4065-9841-d6d59278cbfc None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:22.390166 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08433a96-c578-4065-9841-d6d59278cbfc None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:22.391191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08433a96-c578-4065-9841-d6d59278cbfc None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:22.392624 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08433a96-c578-4065-9841-d6d59278cbfc None admin] RBAC: Adding query filter params (name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:22.395553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08433a96-c578-4065-9841-d6d59278cbfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:22.397171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08433a96-c578-4065-9841-d6d59278cbfc None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:22.410023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 278/548] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:22.416275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08433a96-c578-4065-9841-d6d59278cbfc None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:22.438463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:22.468003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338d7b6-8dd4-4113-999a-40e9db06fae6 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:22.534887 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d30a1-b153-4a13-bc1d-32ff4e6c63ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:22.536149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d30a1-b153-4a13-bc1d-32ff4e6c63ed None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:22.536723 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d30a1-b153-4a13-bc1d-32ff4e6c63ed None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:22.537192 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d30a1-b153-4a13-bc1d-32ff4e6c63ed None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:22.538061 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d30a1-b153-4a13-bc1d-32ff4e6c63ed None admin] RBAC: Adding query filter params (name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:22.538558 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d30a1-b153-4a13-bc1d-32ff4e6c63ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:22.540515 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d30a1-b153-4a13-bc1d-32ff4e6c63ed None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:22.560300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 271/549] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:22.560300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601f14e-8e0d-493c-953a-6eeb427bc2c2 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:22.560300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601f14e-8e0d-493c-953a-6eeb427bc2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:22.560300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601f14e-8e0d-493c-953a-6eeb427bc2c2 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:22.560300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 272/550] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:22.563258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601f14e-8e0d-493c-953a-6eeb427bc2c2 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:22.568805 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:22.592302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08433a96-c578-4065-9841-d6d59278cbfc None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:22.629065 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b2ac1-7ad4-41c0-8547-21ef97265391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:22.629695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b2ac1-7ad4-41c0-8547-21ef97265391 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:22.629981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b2ac1-7ad4-41c0-8547-21ef97265391 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:22.630316 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b2ac1-7ad4-41c0-8547-21ef97265391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:22.641134 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b2ac1-7ad4-41c0-8547-21ef97265391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:22.642593 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b2ac1-7ad4-41c0-8547-21ef97265391 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-221b2ac1-7ad4-41c0-8547-21ef97265391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:22.659215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:22.660255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:22.665601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:22.665601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:22.665601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:22.665601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 279/551] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:22.665601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0defb-2611-48d4-af7e-77cf10997d07 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:22.665601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0defb-2611-48d4-af7e-77cf10997d07 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:22.665601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0defb-2611-48d4-af7e-77cf10997d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:22.684717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-ddc0defb-2611-48d4-af7e-77cf10997d07 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:22.685692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ddc0defb-2611-48d4-af7e-77cf10997d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:22.715482 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601f14e-8e0d-493c-953a-6eeb427bc2c2 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:22.764666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 280/552] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:22.764666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0defb-2611-48d4-af7e-77cf10997d07 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:22.767987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426ef1d-e1a5-400b-b27e-703d45671586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:22.767987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426ef1d-e1a5-400b-b27e-703d45671586 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:22.767987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426ef1d-e1a5-400b-b27e-703d45671586 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:22.767987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426ef1d-e1a5-400b-b27e-703d45671586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:22.789888 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9426ef1d-e1a5-400b-b27e-703d45671586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:22.791271 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9426ef1d-e1a5-400b-b27e-703d45671586 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:22.808063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-9426ef1d-e1a5-400b-b27e-703d45671586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:22.817782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:22.818700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:22.820070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:22.820070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:22.820070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:22.820296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 273/553] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:01:22.825461 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426ef1d-e1a5-400b-b27e-703d45671586 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:22.906106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:22.976426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0defb-2611-48d4-af7e-77cf10997d07 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:23.015289 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426ef1d-e1a5-400b-b27e-703d45671586 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:23.047520 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddde9ab-0977-454d-a8f0-fb8bff1aa4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:23.048048 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddde9ab-0977-454d-a8f0-fb8bff1aa4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:23.048328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddde9ab-0977-454d-a8f0-fb8bff1aa4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:23.048595 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddde9ab-0977-454d-a8f0-fb8bff1aa4b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:23.049264 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddde9ab-0977-454d-a8f0-fb8bff1aa4b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:23.049561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddde9ab-0977-454d-a8f0-fb8bff1aa4b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:23.050975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddde9ab-0977-454d-a8f0-fb8bff1aa4b2 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:23.076916 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b148c61-a7e4-4a7a-98e3-050d7f24122f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:23.077598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b148c61-a7e4-4a7a-98e3-050d7f24122f None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:23.077868 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b148c61-a7e4-4a7a-98e3-050d7f24122f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:23.078140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b148c61-a7e4-4a7a-98e3-050d7f24122f None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:23.078788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b148c61-a7e4-4a7a-98e3-050d7f24122f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:23.079092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b148c61-a7e4-4a7a-98e3-050d7f24122f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:23.079251 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 281/554] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:23.086113 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b148c61-a7e4-4a7a-98e3-050d7f24122f None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:23.086860 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddde9ab-0977-454d-a8f0-fb8bff1aa4b2 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:23.099336 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 274/555] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:23.100179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b148c61-a7e4-4a7a-98e3-050d7f24122f None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:23.162742 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:23.204046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddde9ab-0977-454d-a8f0-fb8bff1aa4b2 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:23.206276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b148c61-a7e4-4a7a-98e3-050d7f24122f None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:23.230180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:23.239390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88c233-048d-4f7c-806f-4509ec3a932d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:23.240013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88c233-048d-4f7c-806f-4509ec3a932d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:23.242032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88c233-048d-4f7c-806f-4509ec3a932d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:23.242032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88c233-048d-4f7c-806f-4509ec3a932d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:23.244423 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f55ef-cb4b-42d2-bc6d-e7a47c119d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:23.245068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f55ef-cb4b-42d2-bc6d-e7a47c119d4b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:23.245349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f55ef-cb4b-42d2-bc6d-e7a47c119d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:23.245615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f55ef-cb4b-42d2-bc6d-e7a47c119d4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:23.248733 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88c233-048d-4f7c-806f-4509ec3a932d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:23.250182 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88c233-048d-4f7c-806f-4509ec3a932d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:23.256193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f55ef-cb4b-42d2-bc6d-e7a47c119d4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-ea5f55ef-cb4b-42d2-bc6d-e7a47c119d4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:23.260893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:23.261840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:23.262628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:23.262628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:23.262628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:23.262628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:23.262628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:23.262628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:23.262628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:23.262628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:23.262628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:23.262628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:23.262628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:23.262628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:23.263930 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 275/556] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-6b88c233-048d-4f7c-806f-4509ec3a932d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:23.263998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:23.265181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:23.266877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:23.266877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:23.266877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:23.266877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:23.266877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:23.266877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:23.266877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:23.266877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:23.266877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 12 12:01:23.266877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:23.268107 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d05c0f-d763-416a-bd12-8b23631b81e1 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:23.268107 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d05c0f-d763-416a-bd12-8b23631b81e1 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:23.268399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 282/557] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:01:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:01:23.268399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88c233-048d-4f7c-806f-4509ec3a932d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:23.269015 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d05c0f-d763-416a-bd12-8b23631b81e1 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:23.271131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 276/558] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:23.272672 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d05c0f-d763-416a-bd12-8b23631b81e1 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:23.376366 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88c233-048d-4f7c-806f-4509ec3a932d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:23.381698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d05c0f-d763-416a-bd12-8b23631b81e1 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:23.406372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:23.414202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ecbc4-e539-4c02-86b5-1d5298b6287c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:23.415507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ecbc4-e539-4c02-86b5-1d5298b6287c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:23.415850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ecbc4-e539-4c02-86b5-1d5298b6287c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:23.416130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ecbc4-e539-4c02-86b5-1d5298b6287c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:23.422862 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d18ad-ed3d-495c-9f42-967a24fa2019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:23.423651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d18ad-ed3d-495c-9f42-967a24fa2019 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:23.423953 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d18ad-ed3d-495c-9f42-967a24fa2019 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:23.424052 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ecbc4-e539-4c02-86b5-1d5298b6287c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:23.424232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d18ad-ed3d-495c-9f42-967a24fa2019 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:23.424931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d18ad-ed3d-495c-9f42-967a24fa2019 None admin] RBAC: Adding query filter params (name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:23.425282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d18ad-ed3d-495c-9f42-967a24fa2019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:23.425519 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ecbc4-e539-4c02-86b5-1d5298b6287c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:23.427211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d18ad-ed3d-495c-9f42-967a24fa2019 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:23.440977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 277/559] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:23.440977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d18ad-ed3d-495c-9f42-967a24fa2019 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-9b9ecbc4-e539-4c02-86b5-1d5298b6287c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:23.441218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:23.442783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:23.443695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:23.443695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:23.443695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:23.443695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:23.443695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:23.443695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:23.443695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:23.443695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:23.443695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:01:23.443695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:23.443695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 283/560] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 12 12:01:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:23.446606 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5215d-1630-4f86-a23a-c6ed6abd3493 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:23.446606 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5215d-1630-4f86-a23a-c6ed6abd3493 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:23.446606 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5215d-1630-4f86-a23a-c6ed6abd3493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:23.464425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-99e5215d-1630-4f86-a23a-c6ed6abd3493 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:23.465641 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-99e5215d-1630-4f86-a23a-c6ed6abd3493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:23.510314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:23.544992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 284/561] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:23.546288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5215d-1630-4f86-a23a-c6ed6abd3493 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:23.556663 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d18ad-ed3d-495c-9f42-967a24fa2019 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:23.591262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf26344-db9f-4a8c-8e50-ad9aaab9ce60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:23.591852 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf26344-db9f-4a8c-8e50-ad9aaab9ce60 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:23.592128 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf26344-db9f-4a8c-8e50-ad9aaab9ce60 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:23.592396 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf26344-db9f-4a8c-8e50-ad9aaab9ce60 None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:23.593151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf26344-db9f-4a8c-8e50-ad9aaab9ce60 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:23.593451 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf26344-db9f-4a8c-8e50-ad9aaab9ce60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:23.594868 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf26344-db9f-4a8c-8e50-ad9aaab9ce60 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:23.612333 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 278/562] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Aug 12 12:01:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:23.618053 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf26344-db9f-4a8c-8e50-ad9aaab9ce60 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:23.654035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5215d-1630-4f86-a23a-c6ed6abd3493 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:23.662101 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:23.690262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebdb66-7ebf-4682-9093-9e86a5369227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:23.690920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebdb66-7ebf-4682-9093-9e86a5369227 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:23.691232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebdb66-7ebf-4682-9093-9e86a5369227 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:23.692659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebdb66-7ebf-4682-9093-9e86a5369227 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:23.702587 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebdb66-7ebf-4682-9093-9e86a5369227 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:23.704130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebdb66-7ebf-4682-9093-9e86a5369227 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-72ebdb66-7ebf-4682-9093-9e86a5369227 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:23.719290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:23.720253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:23.721145 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:23.721145 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:23.721145 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:23.721145 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 285/563] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:23.721797 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b356d3-c6f3-4afe-9cab-22f7b3209883 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:23.721797 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b356d3-c6f3-4afe-9cab-22f7b3209883 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:23.722130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b356d3-c6f3-4afe-9cab-22f7b3209883 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:23.723525 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 286/564] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:23.727357 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf26344-db9f-4a8c-8e50-ad9aaab9ce60 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:23.728009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b356d3-c6f3-4afe-9cab-22f7b3209883 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:23.757299 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:23.765058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ba219-0a03-4c6d-9528-5765b5fc35c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:23.765653 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ba219-0a03-4c6d-9528-5765b5fc35c9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:23.765936 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ba219-0a03-4c6d-9528-5765b5fc35c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:23.766210 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ba219-0a03-4c6d-9528-5765b5fc35c9 None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:23.766962 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ba219-0a03-4c6d-9528-5765b5fc35c9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:23.767264 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ba219-0a03-4c6d-9528-5765b5fc35c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:23.768712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ba219-0a03-4c6d-9528-5765b5fc35c9 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:23.786675 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 279/565] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 12 12:01:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:23.787492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ba219-0a03-4c6d-9528-5765b5fc35c9 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:23.848615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b356d3-c6f3-4afe-9cab-22f7b3209883 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:23.882312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb715f88-15bd-4c8a-869a-fb868eca465c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:23.882921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb715f88-15bd-4c8a-869a-fb868eca465c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:23.883200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb715f88-15bd-4c8a-869a-fb868eca465c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:23.883523 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb715f88-15bd-4c8a-869a-fb868eca465c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:23.886185 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:23.892375 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ba219-0a03-4c6d-9528-5765b5fc35c9 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:23.926668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3493963-5832-40e4-81a3-d8f59ef158ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:23.927342 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3493963-5832-40e4-81a3-d8f59ef158ee None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:23.927622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3493963-5832-40e4-81a3-d8f59ef158ee None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:23.927892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3493963-5832-40e4-81a3-d8f59ef158ee None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:23.928553 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3493963-5832-40e4-81a3-d8f59ef158ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:23.928884 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3493963-5832-40e4-81a3-d8f59ef158ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:23.930514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3493963-5832-40e4-81a3-d8f59ef158ee None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:23.941513 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 280/566] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:23.942605 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5b609-9b0e-4935-8d59-7511d6477f0d None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:23.942886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5b609-9b0e-4935-8d59-7511d6477f0d None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:23.943158 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5b609-9b0e-4935-8d59-7511d6477f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:23.955748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb715f88-15bd-4c8a-869a-fb868eca465c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:23.957339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb715f88-15bd-4c8a-869a-fb868eca465c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:23.959858 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-10c5b609-9b0e-4935-8d59-7511d6477f0d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:23.961067 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-10c5b609-9b0e-4935-8d59-7511d6477f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:23.984618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:24.030449 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 281/567] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:24.031281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5b609-9b0e-4935-8d59-7511d6477f0d None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:24.063919 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 287/568] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:24.065042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb715f88-15bd-4c8a-869a-fb868eca465c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:24.140585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:24.152966 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5b609-9b0e-4935-8d59-7511d6477f0d None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:24.173263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb715f88-15bd-4c8a-869a-fb868eca465c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:24.189509 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3083ec-74ae-4d0e-9970-55a238e0f900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:24.190093 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3083ec-74ae-4d0e-9970-55a238e0f900 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:24.190375 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3083ec-74ae-4d0e-9970-55a238e0f900 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:24.190698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3083ec-74ae-4d0e-9970-55a238e0f900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:24.201703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3083ec-74ae-4d0e-9970-55a238e0f900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:24.202880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:24.204016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3083ec-74ae-4d0e-9970-55a238e0f900 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:24.211114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecec49f-81f7-4c1e-9efa-16c3cad5c925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:24.211809 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecec49f-81f7-4c1e-9efa-16c3cad5c925 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:24.212090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecec49f-81f7-4c1e-9efa-16c3cad5c925 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:24.212360 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecec49f-81f7-4c1e-9efa-16c3cad5c925 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-bd3083ec-74ae-4d0e-9970-55a238e0f900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:24.221876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:24.223471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:24.224594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:24.224594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:24.224594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:24.224835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecec49f-81f7-4c1e-9efa-16c3cad5c925 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-0ecec49f-81f7-4c1e-9efa-16c3cad5c925 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:24.231524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:24.232581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:24.233591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:24.233591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:24.233591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:24.233591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:24.233591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:24.233591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:24.233591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:24.233591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:24.233591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:24.233591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:24.233591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:24.233591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:24.233591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 288/569] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:24.233591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecec49f-81f7-4c1e-9efa-16c3cad5c925 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:24.238688 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 282/570] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:24.238688 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3083ec-74ae-4d0e-9970-55a238e0f900 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:24.335897 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecec49f-81f7-4c1e-9efa-16c3cad5c925 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:24.341828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3083ec-74ae-4d0e-9970-55a238e0f900 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:24.368489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d779c-f3f7-40a2-bd62-1796c74746b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:24.369131 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d779c-f3f7-40a2-bd62-1796c74746b0 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:24.369408 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d779c-f3f7-40a2-bd62-1796c74746b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:24.369679 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d779c-f3f7-40a2-bd62-1796c74746b0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:24.374866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acb99e-1b57-4170-9fb0-805dc9569754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:24.375499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acb99e-1b57-4170-9fb0-805dc9569754 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:24.375763 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acb99e-1b57-4170-9fb0-805dc9569754 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:24.376031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acb99e-1b57-4170-9fb0-805dc9569754 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:24.376585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d779c-f3f7-40a2-bd62-1796c74746b0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:24.378061 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10d779c-f3f7-40a2-bd62-1796c74746b0 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:24.385118 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:24.387268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3acb99e-1b57-4170-9fb0-805dc9569754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:24.388732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3acb99e-1b57-4170-9fb0-805dc9569754 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-d10d779c-f3f7-40a2-bd62-1796c74746b0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:24.390379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:24.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:24.411776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:24.411776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:24.411776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:24.411776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:24.411776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:24.411776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:24.411776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:24.411776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:24.411776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 12 12:01:24.411776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:24.411776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 289/571] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:24.411776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d779c-f3f7-40a2-bd62-1796c74746b0 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-e3acb99e-1b57-4170-9fb0-805dc9569754 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:24.412473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:24.413302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:24.414153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:24.414153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:24.414153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:24.414153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 283/572] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:24.414153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acb99e-1b57-4170-9fb0-805dc9569754 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:24.442720 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:24.492790 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d779c-f3f7-40a2-bd62-1796c74746b0 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:24.505426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acb99e-1b57-4170-9fb0-805dc9569754 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:24.525496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128c45f-aa0f-4816-9f9b-57e22651af69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:24.526087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128c45f-aa0f-4816-9f9b-57e22651af69 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:24.526368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128c45f-aa0f-4816-9f9b-57e22651af69 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:24.526639 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128c45f-aa0f-4816-9f9b-57e22651af69 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:24.527271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128c45f-aa0f-4816-9f9b-57e22651af69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:24.527578 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128c45f-aa0f-4816-9f9b-57e22651af69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:24.528970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128c45f-aa0f-4816-9f9b-57e22651af69 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:24.538715 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675bfe3-f4c4-4dbe-9a51-e8dc4979b9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:24.539305 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675bfe3-f4c4-4dbe-9a51-e8dc4979b9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:24.539587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675bfe3-f4c4-4dbe-9a51-e8dc4979b9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:24.539862 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675bfe3-f4c4-4dbe-9a51-e8dc4979b9b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:24.540247 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 290/573] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:24.540534 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675bfe3-f4c4-4dbe-9a51-e8dc4979b9b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:24.540868 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675bfe3-f4c4-4dbe-9a51-e8dc4979b9b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:24.541056 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128c45f-aa0f-4816-9f9b-57e22651af69 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:24.542223 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675bfe3-f4c4-4dbe-9a51-e8dc4979b9b5 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:24.551683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 284/574] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:24.552344 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675bfe3-f4c4-4dbe-9a51-e8dc4979b9b5 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:24.607882 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:24.642094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128c45f-aa0f-4816-9f9b-57e22651af69 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:24.651643 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:24.651866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675bfe3-f4c4-4dbe-9a51-e8dc4979b9b5 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:24.675209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c7870-4589-40a3-a074-fd3e67984e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:24.675849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c7870-4589-40a3-a074-fd3e67984e30 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:24.676128 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c7870-4589-40a3-a074-fd3e67984e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:24.676398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c7870-4589-40a3-a074-fd3e67984e30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:24.686094 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15885b24-68a4-43c6-99a3-109ee68e8b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:24.686687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15885b24-68a4-43c6-99a3-109ee68e8b52 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:24.686972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15885b24-68a4-43c6-99a3-109ee68e8b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:24.687074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c7870-4589-40a3-a074-fd3e67984e30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:24.687248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15885b24-68a4-43c6-99a3-109ee68e8b52 None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:24.687974 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15885b24-68a4-43c6-99a3-109ee68e8b52 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:24.688280 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15885b24-68a4-43c6-99a3-109ee68e8b52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:24.689726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15885b24-68a4-43c6-99a3-109ee68e8b52 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-fc5c7870-4589-40a3-a074-fd3e67984e30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:24.693271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:24.694191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:24.695200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:24.695200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:24.695200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:24.695200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:24.695200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:24.695200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:24.695200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:24.695200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:24.695200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:24.695200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:24.695200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:24.695200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:24.698354 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 291/575] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:24.699174 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c7870-4589-40a3-a074-fd3e67984e30 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:24.712024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 285/576] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Aug 12 12:01:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:24.712024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15885b24-68a4-43c6-99a3-109ee68e8b52 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:24.797662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c7870-4589-40a3-a074-fd3e67984e30 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:24.806107 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15885b24-68a4-43c6-99a3-109ee68e8b52 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:24.820168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:24.831570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6389bfd-1ef3-4cb1-966e-c6dd55d13640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:24.832706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6389bfd-1ef3-4cb1-966e-c6dd55d13640 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:24.832987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6389bfd-1ef3-4cb1-966e-c6dd55d13640 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:24.834403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6389bfd-1ef3-4cb1-966e-c6dd55d13640 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:24.834403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6389bfd-1ef3-4cb1-966e-c6dd55d13640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:24.834403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6389bfd-1ef3-4cb1-966e-c6dd55d13640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:24.835618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6389bfd-1ef3-4cb1-966e-c6dd55d13640 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:24.841959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d731e-9d12-482d-ba65-8b9043428f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:24.843048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d731e-9d12-482d-ba65-8b9043428f26 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:24.843329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d731e-9d12-482d-ba65-8b9043428f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:24.843600 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d731e-9d12-482d-ba65-8b9043428f26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:24.847031 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 292/577] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:24.848357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6389bfd-1ef3-4cb1-966e-c6dd55d13640 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:24.852688 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d731e-9d12-482d-ba65-8b9043428f26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:24.854607 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d731e-9d12-482d-ba65-8b9043428f26 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:24.867058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-e07d731e-9d12-482d-ba65-8b9043428f26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:24.867058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:24.867058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:24.867058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:24.867058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:24.867058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:24.867058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:24.867058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:24.867058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:24.867058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:24.867058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:24.867058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:24.867058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:24.867058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:24.867058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:24.867915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:24.868813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:24.868813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:24.868813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:24.868813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:24.868813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:24.868813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:24.868813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:24.868813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:24.868813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 286/578] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:24.871108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d731e-9d12-482d-ba65-8b9043428f26 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:24.880807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:24.951256 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6389bfd-1ef3-4cb1-966e-c6dd55d13640 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:24.969340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d731e-9d12-482d-ba65-8b9043428f26 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:24.985476 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981a99a-75d8-4d9d-b2a8-915f3711c6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:24.986092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981a99a-75d8-4d9d-b2a8-915f3711c6a9 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:24.986375 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981a99a-75d8-4d9d-b2a8-915f3711c6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:24.986643 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981a99a-75d8-4d9d-b2a8-915f3711c6a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:24.997005 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981a99a-75d8-4d9d-b2a8-915f3711c6a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:24.998579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981a99a-75d8-4d9d-b2a8-915f3711c6a9 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:25.004936 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d395e84-4c8d-43e6-aa9b-e96b3f1ef8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:25.005526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d395e84-4c8d-43e6-aa9b-e96b3f1ef8eb None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:25.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d395e84-4c8d-43e6-aa9b-e96b3f1ef8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:25.010085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d395e84-4c8d-43e6-aa9b-e96b3f1ef8eb None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:25.010085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d395e84-4c8d-43e6-aa9b-e96b3f1ef8eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:25.010085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d395e84-4c8d-43e6-aa9b-e96b3f1ef8eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:25.010085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d395e84-4c8d-43e6-aa9b-e96b3f1ef8eb None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-f981a99a-75d8-4d9d-b2a8-915f3711c6a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:25.012734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:25.013806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:25.014920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:25.014920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:25.014920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:25.014920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 293/579] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:25.014920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981a99a-75d8-4d9d-b2a8-915f3711c6a9 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:25.016669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 287/580] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:25.017428 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d395e84-4c8d-43e6-aa9b-e96b3f1ef8eb None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:25.054220 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:25.101142 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:25.119526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d395e84-4c8d-43e6-aa9b-e96b3f1ef8eb None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:25.119668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981a99a-75d8-4d9d-b2a8-915f3711c6a9 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:25.152383 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f65d6-fbbb-490b-bf94-5ed45cf210ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:25.152684 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8170-0897-4d4f-947f-284d810be902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:25.153034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f65d6-fbbb-490b-bf94-5ed45cf210ff None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:25.153299 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8170-0897-4d4f-947f-284d810be902 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:25.153391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f65d6-fbbb-490b-bf94-5ed45cf210ff None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:25.153571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f65d6-fbbb-490b-bf94-5ed45cf210ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:25.153672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8170-0897-4d4f-947f-284d810be902 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:25.153845 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8170-0897-4d4f-947f-284d810be902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:25.160566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f65d6-fbbb-490b-bf94-5ed45cf210ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:25.162016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f65d6-fbbb-490b-bf94-5ed45cf210ff None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:25.164691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf8170-0897-4d4f-947f-284d810be902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:25.166122 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf8170-0897-4d4f-947f-284d810be902 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:25.172777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-d33f65d6-fbbb-490b-bf94-5ed45cf210ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:25.172777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:25.172777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:25.172777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:25.172777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:25.172777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:25.172777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:25.172777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:25.172777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:25.172777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:25.172777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:25.172777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:25.172777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:25.172777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:25.172777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:25.173614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:25.174503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:25.174503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:25.174503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:25.174503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:25.174503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:25.174503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:25.174503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:25.174503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:25.174503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 288/581] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:25.174975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f65d6-fbbb-490b-bf94-5ed45cf210ff None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-2fbf8170-0897-4d4f-947f-284d810be902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:25.181367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:25.182312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:25.183177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:25.183177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:25.183177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:25.183177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 294/582] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:25.183390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8170-0897-4d4f-947f-284d810be902 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:25.267449 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:25.275691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f65d6-fbbb-490b-bf94-5ed45cf210ff None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:25.281910 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8170-0897-4d4f-947f-284d810be902 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:25.308202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f51a4-880a-4833-b40f-1f472c00a448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:25.308824 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f51a4-880a-4833-b40f-1f472c00a448 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:25.309108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f51a4-880a-4833-b40f-1f472c00a448 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:25.309381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f51a4-880a-4833-b40f-1f472c00a448 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:25.310070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f51a4-880a-4833-b40f-1f472c00a448 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:25.310378 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f51a4-880a-4833-b40f-1f472c00a448 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:25.310572 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:25.311793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f51a4-880a-4833-b40f-1f472c00a448 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:25.315323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42028580-3f9a-4d41-a84e-27ea62bbaf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:25.319127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42028580-3f9a-4d41-a84e-27ea62bbaf13 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:25.319127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42028580-3f9a-4d41-a84e-27ea62bbaf13 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:25.319127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42028580-3f9a-4d41-a84e-27ea62bbaf13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:25.324552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 289/583] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:25.325510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f51a4-880a-4833-b40f-1f472c00a448 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:25.327726 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42028580-3f9a-4d41-a84e-27ea62bbaf13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:25.329208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42028580-3f9a-4d41-a84e-27ea62bbaf13 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-42028580-3f9a-4d41-a84e-27ea62bbaf13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:25.344415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:25.345397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:25.346240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:25.346240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:25.346240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:25.346240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 295/584] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:25.346466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42028580-3f9a-4d41-a84e-27ea62bbaf13 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:25.427877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f51a4-880a-4833-b40f-1f472c00a448 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:25.446681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42028580-3f9a-4d41-a84e-27ea62bbaf13 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:25.460601 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e983a-55e8-4900-9020-2264f3889e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:25.461222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e983a-55e8-4900-9020-2264f3889e67 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:25.461499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e983a-55e8-4900-9020-2264f3889e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:25.461766 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e983a-55e8-4900-9020-2264f3889e67 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:25.462398 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e983a-55e8-4900-9020-2264f3889e67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:25.462696 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e983a-55e8-4900-9020-2264f3889e67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:25.464044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e983a-55e8-4900-9020-2264f3889e67 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:25.475469 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 290/585] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:25.476225 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e983a-55e8-4900-9020-2264f3889e67 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:25.479200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257da636-709b-4c70-8d5a-3e2c3672d284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:25.479880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257da636-709b-4c70-8d5a-3e2c3672d284 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:25.480164 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257da636-709b-4c70-8d5a-3e2c3672d284 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:25.480435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257da636-709b-4c70-8d5a-3e2c3672d284 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:25.481158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257da636-709b-4c70-8d5a-3e2c3672d284 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:25.481465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257da636-709b-4c70-8d5a-3e2c3672d284 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:25.482880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257da636-709b-4c70-8d5a-3e2c3672d284 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:25.492918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:25.495466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 296/586] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:25.496060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257da636-709b-4c70-8d5a-3e2c3672d284 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:25.542742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:25.574936 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e983a-55e8-4900-9020-2264f3889e67 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:25.595689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257da636-709b-4c70-8d5a-3e2c3672d284 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:25.608480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39467175-04ae-4fa8-bbc2-6cacd40bb3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:25.609107 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39467175-04ae-4fa8-bbc2-6cacd40bb3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:25.609415 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39467175-04ae-4fa8-bbc2-6cacd40bb3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:25.609684 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39467175-04ae-4fa8-bbc2-6cacd40bb3a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:25.610324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39467175-04ae-4fa8-bbc2-6cacd40bb3a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:25.610624 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39467175-04ae-4fa8-bbc2-6cacd40bb3a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:25.611979 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39467175-04ae-4fa8-bbc2-6cacd40bb3a6 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:25.624403 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 291/587] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:25.625219 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39467175-04ae-4fa8-bbc2-6cacd40bb3a6 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:25.635778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf1758-7695-410c-b89d-d95602ca6901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:25.636439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf1758-7695-410c-b89d-d95602ca6901 None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:25.636759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf1758-7695-410c-b89d-d95602ca6901 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:25.637041 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf1758-7695-410c-b89d-d95602ca6901 None admin] PATH_INFO: `/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/687d0375abd44d2481f988c6df90fad1/roles/1416a05bf5ba4de88889c54ff5c58da2` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:25.652322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf1758-7695-410c-b89d-d95602ca6901 None admin] RBAC: Authorizing `identity:create_grant(project_id=20c5570631a34c4ab3868eb1cd96e345, user_id=687d0375abd44d2481f988c6df90fad1, role_id=1416a05bf5ba4de88889c54ff5c58da2)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:25.653814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf1758-7695-410c-b89d-d95602ca6901 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:25.698048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:25.713681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 297/588] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:01:25 2022] PUT /identity/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/687d0375abd44d2481f988c6df90fad1/roles/1416a05bf5ba4de88889c54ff5c58da2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:01:25.714488 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf1758-7695-410c-b89d-d95602ca6901 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:25.723337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39467175-04ae-4fa8-bbc2-6cacd40bb3a6 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:25.765028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8f4b4-ea6c-4357-809d-579ac26084fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:25.765644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8f4b4-ea6c-4357-809d-579ac26084fe None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:25.765921 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8f4b4-ea6c-4357-809d-579ac26084fe None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:25.766190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8f4b4-ea6c-4357-809d-579ac26084fe None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:25.766852 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8f4b4-ea6c-4357-809d-579ac26084fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:25.767155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8f4b4-ea6c-4357-809d-579ac26084fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:25.768566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8f4b4-ea6c-4357-809d-579ac26084fe None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:25.782388 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 292/589] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:25.783079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8f4b4-ea6c-4357-809d-579ac26084fe None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:25.837137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:25.844306 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf1758-7695-410c-b89d-d95602ca6901 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:25.885919 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0f448-ba40-43e7-b132-7c1e56d362e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:25.886549 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0f448-ba40-43e7-b132-7c1e56d362e1 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:25.886835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0f448-ba40-43e7-b132-7c1e56d362e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:25.887112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0f448-ba40-43e7-b132-7c1e56d362e1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:25.894971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a0f448-ba40-43e7-b132-7c1e56d362e1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:25.896734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a0f448-ba40-43e7-b132-7c1e56d362e1 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:25.904567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8f4b4-ea6c-4357-809d-579ac26084fe None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-77a0f448-ba40-43e7-b132-7c1e56d362e1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:25.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:25.912666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:25.913706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:25.913706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:25.913706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:25.913706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:25.913706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:25.913706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:25.913706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:25.913706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:25.913706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 12:01:25.913706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:25.913706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 298/590] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:25.914275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0f448-ba40-43e7-b132-7c1e56d362e1 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:25.946594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737fe12-6473-474d-b810-479119255792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:25.947196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737fe12-6473-474d-b810-479119255792 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:25.947477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737fe12-6473-474d-b810-479119255792 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:25.947749 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737fe12-6473-474d-b810-479119255792 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:25.948475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737fe12-6473-474d-b810-479119255792 None admin] RBAC: Adding query filter params (role.id=1416a05bf5ba4de88889c54ff5c58da2, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=5123401076a84ffcab49b00fee65d14d) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:25.948819 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737fe12-6473-474d-b810-479119255792 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:25.950235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737fe12-6473-474d-b810-479119255792 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:25.952030 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:25.956749 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 293/591] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:25 2022] GET /identity/v3/role_assignments?user.id=5123401076a84ffcab49b00fee65d14d&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=1416a05bf5ba4de88889c54ff5c58da2 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:25.957550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737fe12-6473-474d-b810-479119255792 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:26.046192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0f448-ba40-43e7-b132-7c1e56d362e1 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:26.084233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737fe12-6473-474d-b810-479119255792 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:26.097007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47a8de-5463-4531-b6cb-c111b747d1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:26.098405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47a8de-5463-4531-b6cb-c111b747d1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:26.098695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47a8de-5463-4531-b6cb-c111b747d1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:26.098972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47a8de-5463-4531-b6cb-c111b747d1b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:26.109521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47a8de-5463-4531-b6cb-c111b747d1b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:26.111417 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47a8de-5463-4531-b6cb-c111b747d1b2 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:26.124161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-0f47a8de-5463-4531-b6cb-c111b747d1b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:26.124161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:26.124161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:26.124161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:26.124161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:26.124161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:26.124161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:26.124161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:26.124161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:26.124161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:26.124161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:26.124161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:26.124161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:26.124161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:26.124161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:26.125135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:26.126003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:26.126003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:26.126003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:26.126003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:26.126003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:26.126003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:26.126003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:26.126003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:26.129862 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921be605-cc40-43eb-9ea0-034aec2471df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:26.130602 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 299/592] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:26.131072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921be605-cc40-43eb-9ea0-034aec2471df None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:26.131489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47a8de-5463-4531-b6cb-c111b747d1b2 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:26.131771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921be605-cc40-43eb-9ea0-034aec2471df None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:26.132256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921be605-cc40-43eb-9ea0-034aec2471df None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:26.141159 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921be605-cc40-43eb-9ea0-034aec2471df None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:26.143253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921be605-cc40-43eb-9ea0-034aec2471df None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:26.148443 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-921be605-cc40-43eb-9ea0-034aec2471df None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:26.159047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:26.162471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:26.163641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:26.163641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:26.163641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:26.163641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:26.163641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:26.163641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:26.163641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:26.163641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:26.163641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 12 12:01:26.163641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:26.163641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 294/593] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:26.163641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921be605-cc40-43eb-9ea0-034aec2471df None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:26.244556 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:26.255559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47a8de-5463-4531-b6cb-c111b747d1b2 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:26.291446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921be605-cc40-43eb-9ea0-034aec2471df None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:26.300582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd35712-0c39-4dc5-8930-b37dba49a9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:26.301658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd35712-0c39-4dc5-8930-b37dba49a9dd None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:26.301943 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd35712-0c39-4dc5-8930-b37dba49a9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:26.302214 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd35712-0c39-4dc5-8930-b37dba49a9dd None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:26.303354 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd35712-0c39-4dc5-8930-b37dba49a9dd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:26.303663 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd35712-0c39-4dc5-8930-b37dba49a9dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:26.305661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd35712-0c39-4dc5-8930-b37dba49a9dd None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:26.327816 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 300/594] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Aug 12 12:01:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:26.329048 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd35712-0c39-4dc5-8930-b37dba49a9dd None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:26.331218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41df98-d301-4576-ab11-f1d5a05b9569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:26.332590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41df98-d301-4576-ab11-f1d5a05b9569 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:26.332911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41df98-d301-4576-ab11-f1d5a05b9569 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:26.333196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41df98-d301-4576-ab11-f1d5a05b9569 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:26.334905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41df98-d301-4576-ab11-f1d5a05b9569 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:26.335156 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41df98-d301-4576-ab11-f1d5a05b9569 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:26.336586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41df98-d301-4576-ab11-f1d5a05b9569 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:26.353151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 295/595] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:26.354590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41df98-d301-4576-ab11-f1d5a05b9569 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:26.408419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:26.448285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd35712-0c39-4dc5-8930-b37dba49a9dd None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:26.484183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41df98-d301-4576-ab11-f1d5a05b9569 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:26.499192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd22a2-fc1a-489e-bc93-6fbb20e07a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:26.499937 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd22a2-fc1a-489e-bc93-6fbb20e07a41 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:26.500323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd22a2-fc1a-489e-bc93-6fbb20e07a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:26.500994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd22a2-fc1a-489e-bc93-6fbb20e07a41 None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:26.502311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd22a2-fc1a-489e-bc93-6fbb20e07a41 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:26.502728 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd22a2-fc1a-489e-bc93-6fbb20e07a41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:26.505320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd22a2-fc1a-489e-bc93-6fbb20e07a41 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:26.506530 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:26.525260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61666b5e-4378-4288-9bf5-d08426d642f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:26.526159 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61666b5e-4378-4288-9bf5-d08426d642f2 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:26.526575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61666b5e-4378-4288-9bf5-d08426d642f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:26.526974 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61666b5e-4378-4288-9bf5-d08426d642f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:26.533135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 301/596] 10.222.0.27 () {56 vars in 1139 bytes} [Fri Aug 12 12:01:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:26.533135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd22a2-fc1a-489e-bc93-6fbb20e07a41 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:26.542582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61666b5e-4378-4288-9bf5-d08426d642f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:26.544567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61666b5e-4378-4288-9bf5-d08426d642f2 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-61666b5e-4378-4288-9bf5-d08426d642f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:26.561633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:26.562693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:26.563756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:26.563756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:26.563756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:26.565716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 296/597] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:26.566778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61666b5e-4378-4288-9bf5-d08426d642f2 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:26.661750 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd22a2-fc1a-489e-bc93-6fbb20e07a41 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:26.675139 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61666b5e-4378-4288-9bf5-d08426d642f2 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:26.699107 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:26.706471 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9247911-f5eb-46e4-bbab-59b7545ea6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:26.707567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9247911-f5eb-46e4-bbab-59b7545ea6be None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:26.707878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9247911-f5eb-46e4-bbab-59b7545ea6be None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:26.708159 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9247911-f5eb-46e4-bbab-59b7545ea6be None admin] PATH_INFO: `/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/ffe9f3fec7104a289e05e88e2d751458/roles/1416a05bf5ba4de88889c54ff5c58da2` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:26.718307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0fbe6-f519-4088-8977-c587313d626e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:26.720249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0fbe6-f519-4088-8977-c587313d626e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:26.720249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0fbe6-f519-4088-8977-c587313d626e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:26.720249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0fbe6-f519-4088-8977-c587313d626e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:26.724708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9247911-f5eb-46e4-bbab-59b7545ea6be None admin] RBAC: Authorizing `identity:create_grant(project_id=20c5570631a34c4ab3868eb1cd96e345, user_id=ffe9f3fec7104a289e05e88e2d751458, role_id=1416a05bf5ba4de88889c54ff5c58da2)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:26.726175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9247911-f5eb-46e4-bbab-59b7545ea6be None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:26.733762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0fbe6-f519-4088-8977-c587313d626e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:26.735214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0fbe6-f519-4088-8977-c587313d626e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-53e0fbe6-f519-4088-8977-c587313d626e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:26.750719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:26.751647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:26.753206 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:26.753206 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:26.753206 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:26.753206 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 297/598] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:26.753206 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0fbe6-f519-4088-8977-c587313d626e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:26.780368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:26.807334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 302/599] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:01:26 2022] PUT /identity/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/ffe9f3fec7104a289e05e88e2d751458/roles/1416a05bf5ba4de88889c54ff5c58da2 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:01:26.808187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9247911-f5eb-46e4-bbab-59b7545ea6be None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:26.905830 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0fbe6-f519-4088-8977-c587313d626e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:26.945180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53f8cd-b59d-41a7-8e6c-a5463fb00ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:26.946277 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53f8cd-b59d-41a7-8e6c-a5463fb00ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:26.946804 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53f8cd-b59d-41a7-8e6c-a5463fb00ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:26.947269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53f8cd-b59d-41a7-8e6c-a5463fb00ca9 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:26.948189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53f8cd-b59d-41a7-8e6c-a5463fb00ca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:26.948732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53f8cd-b59d-41a7-8e6c-a5463fb00ca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:26.950419 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53f8cd-b59d-41a7-8e6c-a5463fb00ca9 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:26.964729 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 298/600] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:26.974217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53f8cd-b59d-41a7-8e6c-a5463fb00ca9 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:27.034601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9247911-f5eb-46e4-bbab-59b7545ea6be None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:27.073556 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca5e82-0e34-419e-b4fd-76a53ff0fd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:27.074227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca5e82-0e34-419e-b4fd-76a53ff0fd3c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:27.074511 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca5e82-0e34-419e-b4fd-76a53ff0fd3c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:27.074786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca5e82-0e34-419e-b4fd-76a53ff0fd3c None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:27.075435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca5e82-0e34-419e-b4fd-76a53ff0fd3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:27.075736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca5e82-0e34-419e-b4fd-76a53ff0fd3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:27.077172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca5e82-0e34-419e-b4fd-76a53ff0fd3c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:27.090252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 303/601] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:27.092554 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:27.097421 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca5e82-0e34-419e-b4fd-76a53ff0fd3c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:27.117420 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53f8cd-b59d-41a7-8e6c-a5463fb00ca9 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:27.163124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:27.174000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6559c9-e844-4d9b-a1e7-b00844901f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:27.177122 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6559c9-e844-4d9b-a1e7-b00844901f45 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:27.177122 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6559c9-e844-4d9b-a1e7-b00844901f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:27.178652 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6559c9-e844-4d9b-a1e7-b00844901f45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:27.193458 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6559c9-e844-4d9b-a1e7-b00844901f45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:27.194998 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6559c9-e844-4d9b-a1e7-b00844901f45 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:27.220562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-fe6559c9-e844-4d9b-a1e7-b00844901f45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:27.220562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:27.220562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:27.220562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:27.220562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:27.220562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:27.220562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:27.220562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:27.220562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:27.220562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:27.220562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:27.220562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:27.220562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:27.220562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:27.220562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:27.221532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:27.223268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:27.223268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:27.223268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:27.223268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:27.223268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:27.223268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:27.223268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:27.223268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:27.229804 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 299/602] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:27.231546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6559c9-e844-4d9b-a1e7-b00844901f45 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:27.271013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca5e82-0e34-419e-b4fd-76a53ff0fd3c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:27.360049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6559c9-e844-4d9b-a1e7-b00844901f45 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:27.362301 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa360cb-6040-4dd1-aa1c-84f465cada60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:27.363532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa360cb-6040-4dd1-aa1c-84f465cada60 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:27.363915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa360cb-6040-4dd1-aa1c-84f465cada60 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:27.364205 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa360cb-6040-4dd1-aa1c-84f465cada60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:27.381796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa360cb-6040-4dd1-aa1c-84f465cada60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:27.383275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa360cb-6040-4dd1-aa1c-84f465cada60 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:27.396446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a4253-0ba3-45d7-9ca2-8bba8e24e113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:27.397083 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a4253-0ba3-45d7-9ca2-8bba8e24e113 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:27.397364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a4253-0ba3-45d7-9ca2-8bba8e24e113 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:27.397636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a4253-0ba3-45d7-9ca2-8bba8e24e113 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:27.398332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a4253-0ba3-45d7-9ca2-8bba8e24e113 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:27.398635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a4253-0ba3-45d7-9ca2-8bba8e24e113 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:27.400109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a4253-0ba3-45d7-9ca2-8bba8e24e113 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:27.410849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-caa360cb-6040-4dd1-aa1c-84f465cada60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:27.410849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:27.410849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:27.410849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:27.410849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:27.410849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:27.410849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:27.410849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:27.410849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:27.410849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:27.410849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:27.410849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:27.410849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:27.410849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:27.410849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:27.412001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:27.412909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:27.412909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:27.412909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:27.412909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:27.412909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:27.412909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:27.412909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:27.412909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:27.418432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 300/603] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:27.419496 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a4253-0ba3-45d7-9ca2-8bba8e24e113 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:27.420965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 304/604] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:27.421841 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa360cb-6040-4dd1-aa1c-84f465cada60 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:27.422544 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:27.526236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a4253-0ba3-45d7-9ca2-8bba8e24e113 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:27.533842 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa360cb-6040-4dd1-aa1c-84f465cada60 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:27.537088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:27.563249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67b3c9-bbce-4644-8ced-44f89f03a30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:27.563856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67b3c9-bbce-4644-8ced-44f89f03a30a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:27.564130 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67b3c9-bbce-4644-8ced-44f89f03a30a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:27.564407 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67b3c9-bbce-4644-8ced-44f89f03a30a None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:27.565135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c67b3c9-bbce-4644-8ced-44f89f03a30a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:27.565544 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c67b3c9-bbce-4644-8ced-44f89f03a30a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:27.567510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c67b3c9-bbce-4644-8ced-44f89f03a30a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:27.569582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c329d-aec5-477f-8008-ba167d1ff000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:27.570244 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c329d-aec5-477f-8008-ba167d1ff000 None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:27.570524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c329d-aec5-477f-8008-ba167d1ff000 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:27.570796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c329d-aec5-477f-8008-ba167d1ff000 None admin] PATH_INFO: `/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/935ab3a263ae46fcb929029a147a1010/roles/1416a05bf5ba4de88889c54ff5c58da2` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:27.582097 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 301/605] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:27.591416 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67b3c9-bbce-4644-8ced-44f89f03a30a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:27.591584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c329d-aec5-477f-8008-ba167d1ff000 None admin] RBAC: Authorizing `identity:create_grant(project_id=20c5570631a34c4ab3868eb1cd96e345, user_id=935ab3a263ae46fcb929029a147a1010, role_id=1416a05bf5ba4de88889c54ff5c58da2)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:27.593041 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c329d-aec5-477f-8008-ba167d1ff000 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:27.661370 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:27.686017 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 305/606] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:01:27 2022] PUT /identity/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/935ab3a263ae46fcb929029a147a1010/roles/1416a05bf5ba4de88889c54ff5c58da2 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:01:27.703368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67b3c9-bbce-4644-8ced-44f89f03a30a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:27.750530 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e24c1-f94a-48e3-a4a3-7221336297c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:27.751297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e24c1-f94a-48e3-a4a3-7221336297c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:27.751579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e24c1-f94a-48e3-a4a3-7221336297c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:27.751854 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e24c1-f94a-48e3-a4a3-7221336297c6 None admin] PATH_INFO: `/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/745fad9c109a4280906eb0349849117d/roles/1416a05bf5ba4de88889c54ff5c58da2` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:27.774493 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e24c1-f94a-48e3-a4a3-7221336297c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=20c5570631a34c4ab3868eb1cd96e345, user_id=745fad9c109a4280906eb0349849117d, role_id=1416a05bf5ba4de88889c54ff5c58da2)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:27.776048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e24c1-f94a-48e3-a4a3-7221336297c6 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:27.803165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd6b51-fb0d-409e-b36a-b21c43cf4044 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:27.803469 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd6b51-fb0d-409e-b36a-b21c43cf4044 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:27.803634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd6b51-fb0d-409e-b36a-b21c43cf4044 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:27.811883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 306/607] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:27.818325 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd172092-2a8c-4299-8435-c4c5e68cc6ce None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:27.818688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd172092-2a8c-4299-8435-c4c5e68cc6ce None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:27.819039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd172092-2a8c-4299-8435-c4c5e68cc6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:27.841064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-bd172092-2a8c-4299-8435-c4c5e68cc6ce None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:27.842346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-bd172092-2a8c-4299-8435-c4c5e68cc6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:27.845886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 302/608] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:01:27 2022] PUT /identity/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/745fad9c109a4280906eb0349849117d/roles/1416a05bf5ba4de88889c54ff5c58da2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:01:27.927091 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 307/609] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:28.145746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067986d-6e6e-4751-a3f5-8c8d336465b6 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:28.146086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067986d-6e6e-4751-a3f5-8c8d336465b6 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:28.146276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067986d-6e6e-4751-a3f5-8c8d336465b6 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:28.147765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 303/610] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:28.153395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1f91e-b84d-4b82-87d2-84c9c8b0cd96 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:28.153917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1f91e-b84d-4b82-87d2-84c9c8b0cd96 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:28.154387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1f91e-b84d-4b82-87d2-84c9c8b0cd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:28.173430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-41f1f91e-b84d-4b82-87d2-84c9c8b0cd96 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:28.174920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-41f1f91e-b84d-4b82-87d2-84c9c8b0cd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:28.250785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067986d-6e6e-4751-a3f5-8c8d336465b6 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:28.270158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 308/611] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:28.383359 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067986d-6e6e-4751-a3f5-8c8d336465b6 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:28.400733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:28.430649 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b90684-ae97-4479-9ce6-a2debb5b7df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:28.431278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b90684-ae97-4479-9ce6-a2debb5b7df2 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:28.431564 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b90684-ae97-4479-9ce6-a2debb5b7df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:28.431840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b90684-ae97-4479-9ce6-a2debb5b7df2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:28.447323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b90684-ae97-4479-9ce6-a2debb5b7df2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-56b90684-ae97-4479-9ce6-a2debb5b7df2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:28.452368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:28.453269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:28.454077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:28.454077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:28.454077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:28.454077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:28.454077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:28.454077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:28.454077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:28.454077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:28.454077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:28.454077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:28.454077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:28.454077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:28.454077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 304/612] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:28.461978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1f91e-b84d-4b82-87d2-84c9c8b0cd96 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:28.484611 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:28.612385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1f91e-b84d-4b82-87d2-84c9c8b0cd96 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:28.664312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d16496-aac5-4f56-a51f-4bcc0305da19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:28.664870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d16496-aac5-4f56-a51f-4bcc0305da19 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:28.665151 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d16496-aac5-4f56-a51f-4bcc0305da19 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:28.665420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d16496-aac5-4f56-a51f-4bcc0305da19 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:28.666090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d16496-aac5-4f56-a51f-4bcc0305da19 None admin] RBAC: Adding query filter params (name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:28.666391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d16496-aac5-4f56-a51f-4bcc0305da19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:28.667814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d16496-aac5-4f56-a51f-4bcc0305da19 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:28.676426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 309/613] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:28.682199 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b90684-ae97-4479-9ce6-a2debb5b7df2 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:28.799232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b90684-ae97-4479-9ce6-a2debb5b7df2 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:28.819032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d16496-aac5-4f56-a51f-4bcc0305da19 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:28.839659 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e98e5-d2e8-40bf-b714-a206cb9838ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:28.840685 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e98e5-d2e8-40bf-b714-a206cb9838ff None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:28.840978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e98e5-d2e8-40bf-b714-a206cb9838ff None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:28.841251 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e98e5-d2e8-40bf-b714-a206cb9838ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:28.852828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e98e5-d2e8-40bf-b714-a206cb9838ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:28.854631 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e98e5-d2e8-40bf-b714-a206cb9838ff None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-412e98e5-d2e8-40bf-b714-a206cb9838ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:28.872424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:28.873393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:28.874247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:28.874247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:28.874247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:28.874247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 305/614] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:28.879966 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e98e5-d2e8-40bf-b714-a206cb9838ff None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:28.908578 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:28.979779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:29.027649 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d16496-aac5-4f56-a51f-4bcc0305da19 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:29.060959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e98e5-d2e8-40bf-b714-a206cb9838ff None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:29.085915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec667d8-13c1-44b1-acc2-0bf58bd064a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:29.087911 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec667d8-13c1-44b1-acc2-0bf58bd064a3 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:29.088201 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec667d8-13c1-44b1-acc2-0bf58bd064a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:29.088478 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec667d8-13c1-44b1-acc2-0bf58bd064a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:29.109451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec667d8-13c1-44b1-acc2-0bf58bd064a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-3ec667d8-13c1-44b1-acc2-0bf58bd064a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:29.115036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:29.115949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:29.116762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:29.116762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:29.116762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:29.116762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:29.116762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:29.116762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:29.116762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:29.116762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:29.116762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:29.116762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:29.116762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:29.116762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:29.121994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 310/615] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:29.125827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351dbeb1-5570-4d9d-a378-1e5d46af657a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:29.126635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dbeb1-5570-4d9d-a378-1e5d46af657a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:29.126807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dbeb1-5570-4d9d-a378-1e5d46af657a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:29.127030 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dbeb1-5570-4d9d-a378-1e5d46af657a None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:29.127673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351dbeb1-5570-4d9d-a378-1e5d46af657a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:29.127975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351dbeb1-5570-4d9d-a378-1e5d46af657a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:29.129910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351dbeb1-5570-4d9d-a378-1e5d46af657a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:29.130944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec667d8-13c1-44b1-acc2-0bf58bd064a3 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:29.156814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 306/616] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:29.163381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351dbeb1-5570-4d9d-a378-1e5d46af657a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:29.311302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec667d8-13c1-44b1-acc2-0bf58bd064a3 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:29.323380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:29.350745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351dbeb1-5570-4d9d-a378-1e5d46af657a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:29.359005 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:29.365186 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db1608-613b-4b5e-9e45-36e038db159f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:29.366178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db1608-613b-4b5e-9e45-36e038db159f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:29.366736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db1608-613b-4b5e-9e45-36e038db159f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:29.367016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db1608-613b-4b5e-9e45-36e038db159f None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:29.367693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db1608-613b-4b5e-9e45-36e038db159f None admin] RBAC: Adding query filter params (name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:29.368159 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db1608-613b-4b5e-9e45-36e038db159f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:29.370204 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db1608-613b-4b5e-9e45-36e038db159f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:29.384730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 311/617] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:29.386031 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f3bbb-d57e-4a21-a455-9b55f69c4e32 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:29.386318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f3bbb-d57e-4a21-a455-9b55f69c4e32 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:29.386592 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f3bbb-d57e-4a21-a455-9b55f69c4e32 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:29.388021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 312/618] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:29.389272 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f3bbb-d57e-4a21-a455-9b55f69c4e32 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:29.402248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694d187-f789-49ae-bb84-1489b8349594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:29.403402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694d187-f789-49ae-bb84-1489b8349594 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:29.403829 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694d187-f789-49ae-bb84-1489b8349594 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:29.404245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694d187-f789-49ae-bb84-1489b8349594 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:29.415609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694d187-f789-49ae-bb84-1489b8349594 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:29.417641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1694d187-f789-49ae-bb84-1489b8349594 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-1694d187-f789-49ae-bb84-1489b8349594 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:29.434207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:29.435147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:29.436117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:29.436117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:29.436117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:29.436117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:29.436117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:29.436117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:29.436117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:29.436117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:29.436117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:01:29.436117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:29.439329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 307/619] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 12 12:01:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:01:29.440874 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154e844-d8b5-4178-b22f-e1c7f5656d08 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:29.441337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154e844-d8b5-4178-b22f-e1c7f5656d08 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:29.441756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154e844-d8b5-4178-b22f-e1c7f5656d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:29.465518 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-b154e844-d8b5-4178-b22f-e1c7f5656d08 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:29.467537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b154e844-d8b5-4178-b22f-e1c7f5656d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:29.508524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f3bbb-d57e-4a21-a455-9b55f69c4e32 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:29.548393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27772ee-7be0-4c23-a105-926cd4909117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:29.549208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27772ee-7be0-4c23-a105-926cd4909117 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:29.549647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27772ee-7be0-4c23-a105-926cd4909117 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:29.550001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27772ee-7be0-4c23-a105-926cd4909117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:29.560315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 308/620] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:29.560315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154e844-d8b5-4178-b22f-e1c7f5656d08 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:29.575910 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27772ee-7be0-4c23-a105-926cd4909117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:29.577331 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27772ee-7be0-4c23-a105-926cd4909117 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-b27772ee-7be0-4c23-a105-926cd4909117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:29.606110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:29.607094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:29.607959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:29.607959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:29.607959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:29.608362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 313/621] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:29.616651 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27772ee-7be0-4c23-a105-926cd4909117 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:29.700047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:29.703296 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:29.750106 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154e844-d8b5-4178-b22f-e1c7f5656d08 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:29.800731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27772ee-7be0-4c23-a105-926cd4909117 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:29.816674 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fe20b-3904-4024-9bc1-c2f06873f3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:29.816674 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fe20b-3904-4024-9bc1-c2f06873f3ac None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:29.816674 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fe20b-3904-4024-9bc1-c2f06873f3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:29.816674 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fe20b-3904-4024-9bc1-c2f06873f3ac None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:29.817059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035fe20b-3904-4024-9bc1-c2f06873f3ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:29.817059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035fe20b-3904-4024-9bc1-c2f06873f3ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:29.818343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035fe20b-3904-4024-9bc1-c2f06873f3ac None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:29.853965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 309/622] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 12 12:01:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:29.858266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fe20b-3904-4024-9bc1-c2f06873f3ac None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:29.876447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6c04a-9d1c-48d1-a0ad-91774f20f9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:29.877086 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6c04a-9d1c-48d1-a0ad-91774f20f9dc None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:29.877371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6c04a-9d1c-48d1-a0ad-91774f20f9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:29.877649 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6c04a-9d1c-48d1-a0ad-91774f20f9dc None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:29.878334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6c04a-9d1c-48d1-a0ad-91774f20f9dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:29.878925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6c04a-9d1c-48d1-a0ad-91774f20f9dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:29.881148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6c04a-9d1c-48d1-a0ad-91774f20f9dc None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:29.905926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 314/623] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:29.911962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6c04a-9d1c-48d1-a0ad-91774f20f9dc None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:30.010515 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fe20b-3904-4024-9bc1-c2f06873f3ac None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:30.027210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6c04a-9d1c-48d1-a0ad-91774f20f9dc None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:30.037241 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:30.050754 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:30.057989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d097b6-5b44-4f18-9ab9-04d93c241867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:30.059078 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d097b6-5b44-4f18-9ab9-04d93c241867 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:30.059688 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d097b6-5b44-4f18-9ab9-04d93c241867 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:30.060137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d097b6-5b44-4f18-9ab9-04d93c241867 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:30.068992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b40681-08ac-45ba-936e-23ecbc8c6305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:30.069621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b40681-08ac-45ba-936e-23ecbc8c6305 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:30.073389 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b40681-08ac-45ba-936e-23ecbc8c6305 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:30.073389 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b40681-08ac-45ba-936e-23ecbc8c6305 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:30.077439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b40681-08ac-45ba-936e-23ecbc8c6305 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:30.078938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b40681-08ac-45ba-936e-23ecbc8c6305 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-39b40681-08ac-45ba-936e-23ecbc8c6305 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:30.091195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:30.092254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:30.093268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:30.093268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:30.093268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:30.093268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:30.093268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:30.093268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:30.093268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:30.093268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:30.093268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 12 12:01:30.093268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:30.093268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 315/624] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:01:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:30.093268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b40681-08ac-45ba-936e-23ecbc8c6305 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:30.137353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d097b6-5b44-4f18-9ab9-04d93c241867 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:30.138741 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d097b6-5b44-4f18-9ab9-04d93c241867 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:30.197096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b40681-08ac-45ba-936e-23ecbc8c6305 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:30.231180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8c3a1-c7f5-485b-a1de-89a0ab2e1f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:30.231803 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8c3a1-c7f5-485b-a1de-89a0ab2e1f9e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:30.232084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8c3a1-c7f5-485b-a1de-89a0ab2e1f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:30.232366 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8c3a1-c7f5-485b-a1de-89a0ab2e1f9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:30.237737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 310/625] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:30.239471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75c89a-a2da-4cec-bad3-a539dd4a87e7 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:30.239760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75c89a-a2da-4cec-bad3-a539dd4a87e7 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:30.240034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75c89a-a2da-4cec-bad3-a539dd4a87e7 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:30.241463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 311/626] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:30.242575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c24c7-126e-4d6c-9fe4-8d0c4e61310b None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:30.242863 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c24c7-126e-4d6c-9fe4-8d0c4e61310b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:30.243135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c24c7-126e-4d6c-9fe4-8d0c4e61310b None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:30.248066 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 312/627] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:30.248066 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c24c7-126e-4d6c-9fe4-8d0c4e61310b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:30.248250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8c3a1-c7f5-485b-a1de-89a0ab2e1f9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-d5b8c3a1-c7f5-485b-a1de-89a0ab2e1f9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:30.252046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:30.253121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:30.254135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:30.254135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:30.254135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:30.254135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:30.254135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:30.254135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:30.254135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:30.254135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:30.254135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:30.254135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:30.254135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:30.254135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:30.254135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 316/628] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:30.254814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8c3a1-c7f5-485b-a1de-89a0ab2e1f9e None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:30.278993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:30.283394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:30.350738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c24c7-126e-4d6c-9fe4-8d0c4e61310b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:30.356885 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8c3a1-c7f5-485b-a1de-89a0ab2e1f9e None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:30.384405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed781cd-2eb5-4037-a1d2-81b12845532e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:30.385031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed781cd-2eb5-4037-a1d2-81b12845532e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:30.385313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed781cd-2eb5-4037-a1d2-81b12845532e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:30.385591 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed781cd-2eb5-4037-a1d2-81b12845532e None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:30.386348 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed781cd-2eb5-4037-a1d2-81b12845532e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:30.386658 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed781cd-2eb5-4037-a1d2-81b12845532e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:30.388132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed781cd-2eb5-4037-a1d2-81b12845532e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:30.390168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207ad69-9b4c-4e67-a90c-44ed7ddf5781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:30.390787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207ad69-9b4c-4e67-a90c-44ed7ddf5781 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:30.391071 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207ad69-9b4c-4e67-a90c-44ed7ddf5781 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:30.391346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207ad69-9b4c-4e67-a90c-44ed7ddf5781 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:30.403008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207ad69-9b4c-4e67-a90c-44ed7ddf5781 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:30.404451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207ad69-9b4c-4e67-a90c-44ed7ddf5781 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:30.405312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 313/629] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Aug 12 12:01:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:30.410424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280ea78-ac8a-4097-8bec-3d7c300bdb85 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:30.410424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280ea78-ac8a-4097-8bec-3d7c300bdb85 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:30.410424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280ea78-ac8a-4097-8bec-3d7c300bdb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-4207ad69-9b4c-4e67-a90c-44ed7ddf5781 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:30.423727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:30.424739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:30.426049 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:30.426049 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:30.426049 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:30.426237 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-2280ea78-ac8a-4097-8bec-3d7c300bdb85 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:30.426308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 317/630] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:30.426374 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2280ea78-ac8a-4097-8bec-3d7c300bdb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:30.426451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c9aef-5efa-4055-9625-e2268558d0e4 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:30.426657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c9aef-5efa-4055-9625-e2268558d0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:30.426935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c9aef-5efa-4055-9625-e2268558d0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:30.445351 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-4e5c9aef-5efa-4055-9625-e2268558d0e4 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:30.446542 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4e5c9aef-5efa-4055-9625-e2268558d0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:30.492801 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 314/631] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:30.493640 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280ea78-ac8a-4097-8bec-3d7c300bdb85 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:30.514990 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:30.516818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 318/632] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:30.516818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c9aef-5efa-4055-9625-e2268558d0e4 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:30.538550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:30.607514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280ea78-ac8a-4097-8bec-3d7c300bdb85 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:30.637666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c9aef-5efa-4055-9625-e2268558d0e4 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:30.642969 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da55bb3-8bba-4383-b875-17b09dfb7d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:30.643511 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da55bb3-8bba-4383-b875-17b09dfb7d28 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:30.643945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da55bb3-8bba-4383-b875-17b09dfb7d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:30.644193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da55bb3-8bba-4383-b875-17b09dfb7d28 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:30.644881 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da55bb3-8bba-4383-b875-17b09dfb7d28 None admin] RBAC: Adding query filter params (name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:30.645187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da55bb3-8bba-4383-b875-17b09dfb7d28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:30.646570 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da55bb3-8bba-4383-b875-17b09dfb7d28 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:30.655638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 315/633] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:30.656510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da55bb3-8bba-4383-b875-17b09dfb7d28 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:30.676044 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd5971-3f0b-418a-9037-f704de898fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:30.676937 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd5971-3f0b-418a-9037-f704de898fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:30.677422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd5971-3f0b-418a-9037-f704de898fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:30.677920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd5971-3f0b-418a-9037-f704de898fa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:30.689135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd5971-3f0b-418a-9037-f704de898fa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:30.691757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd5971-3f0b-418a-9037-f704de898fa8 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-7cbd5971-3f0b-418a-9037-f704de898fa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:30.707560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:30.708698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:30.709726 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:30.709726 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:30.709726 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:30.711402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 319/634] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:30.712405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd5971-3f0b-418a-9037-f704de898fa8 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:30.748847 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:30.769312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da55bb3-8bba-4383-b875-17b09dfb7d28 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:30.779625 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:30.807482 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0c45a-207f-41f0-941e-5c65d9a7518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:30.808181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0c45a-207f-41f0-941e-5c65d9a7518f None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:30.808531 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0c45a-207f-41f0-941e-5c65d9a7518f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:30.808836 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0c45a-207f-41f0-941e-5c65d9a7518f None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:30.809526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0c45a-207f-41f0-941e-5c65d9a7518f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:30.809768 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0c45a-207f-41f0-941e-5c65d9a7518f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:30.811121 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0c45a-207f-41f0-941e-5c65d9a7518f None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:30.823395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 316/635] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:30.824189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0c45a-207f-41f0-941e-5c65d9a7518f None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:30.830719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd5971-3f0b-418a-9037-f704de898fa8 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:30.863891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34578bf5-5d13-442b-a15d-6fe4f32a2528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:30.864518 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34578bf5-5d13-442b-a15d-6fe4f32a2528 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:30.864834 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34578bf5-5d13-442b-a15d-6fe4f32a2528 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:30.865108 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34578bf5-5d13-442b-a15d-6fe4f32a2528 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:30.875462 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34578bf5-5d13-442b-a15d-6fe4f32a2528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:30.877041 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34578bf5-5d13-442b-a15d-6fe4f32a2528 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-34578bf5-5d13-442b-a15d-6fe4f32a2528 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:30.891435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:30.892358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:30.893735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:30.893735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:30.893735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:30.893735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 320/636] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:30.893735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34578bf5-5d13-442b-a15d-6fe4f32a2528 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:30.920449 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0c45a-207f-41f0-941e-5c65d9a7518f None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:30.953843 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272d9b5-08e7-434e-8f8c-e246b37857ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:30.954437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272d9b5-08e7-434e-8f8c-e246b37857ca None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:30.954717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272d9b5-08e7-434e-8f8c-e246b37857ca None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:30.954989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272d9b5-08e7-434e-8f8c-e246b37857ca None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:30.955623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272d9b5-08e7-434e-8f8c-e246b37857ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:30.955923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272d9b5-08e7-434e-8f8c-e246b37857ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:30.957913 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272d9b5-08e7-434e-8f8c-e246b37857ca None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:30.968890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 317/637] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:30.969679 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272d9b5-08e7-434e-8f8c-e246b37857ca None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:30.972399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:30.996563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34578bf5-5d13-442b-a15d-6fe4f32a2528 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:31.009615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:31.030049 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157483bb-7c11-4c65-8b37-24d503a8bae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:31.030735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157483bb-7c11-4c65-8b37-24d503a8bae5 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:31.031022 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157483bb-7c11-4c65-8b37-24d503a8bae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:31.031293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157483bb-7c11-4c65-8b37-24d503a8bae5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:31.041978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157483bb-7c11-4c65-8b37-24d503a8bae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-157483bb-7c11-4c65-8b37-24d503a8bae5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:31.047124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:31.048202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:31.049222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:31.049222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:31.049222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:31.049222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:31.049222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:31.049222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:31.049222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:31.049222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:31.049222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:31.049222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:31.049222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:31.049222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:31.052783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 321/638] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:31.053994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157483bb-7c11-4c65-8b37-24d503a8bae5 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:31.069553 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272d9b5-08e7-434e-8f8c-e246b37857ca None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:31.102615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa7c6f-bfaa-41a7-9b6d-b62ca4ae7667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:31.103321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa7c6f-bfaa-41a7-9b6d-b62ca4ae7667 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:31.103607 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa7c6f-bfaa-41a7-9b6d-b62ca4ae7667 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:31.103877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa7c6f-bfaa-41a7-9b6d-b62ca4ae7667 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:31.114292 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa7c6f-bfaa-41a7-9b6d-b62ca4ae7667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-3cfa7c6f-bfaa-41a7-9b6d-b62ca4ae7667 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:31.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:31.120472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:31.125806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:31.125806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:31.125806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:31.125806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:31.125806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:31.125806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:31.125806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:31.125806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:31.125806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:31.125806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:31.125806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:31.125806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:31.125806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 318/639] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:31.125806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa7c6f-bfaa-41a7-9b6d-b62ca4ae7667 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:31.152823 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157483bb-7c11-4c65-8b37-24d503a8bae5 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:31.183403 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:31.186884 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62438bdd-252b-4488-8c50-f68e0a1a119e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:31.187506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62438bdd-252b-4488-8c50-f68e0a1a119e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:31.187785 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62438bdd-252b-4488-8c50-f68e0a1a119e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:31.188057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62438bdd-252b-4488-8c50-f68e0a1a119e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:31.195574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62438bdd-252b-4488-8c50-f68e0a1a119e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:31.197066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62438bdd-252b-4488-8c50-f68e0a1a119e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:31.208023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-62438bdd-252b-4488-8c50-f68e0a1a119e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:31.208023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:31.208023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:31.208023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:31.208023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:31.208023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:31.208023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:31.208023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:31.208023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:31.208023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:31.208023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:31.208023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:31.208023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:31.208023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:31.208023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:31.209167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:31.210137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:31.210137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:31.210137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:31.210137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:31.210137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:31.210137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:31.210137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:31.210137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:31.210137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 322/640] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:31.210636 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62438bdd-252b-4488-8c50-f68e0a1a119e None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:31.224883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa7c6f-bfaa-41a7-9b6d-b62ca4ae7667 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:31.241693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:31.258348 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1fe24-2b6d-4bb5-a30e-941636d0371f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:31.258967 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1fe24-2b6d-4bb5-a30e-941636d0371f None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:31.259244 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1fe24-2b6d-4bb5-a30e-941636d0371f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:31.259510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1fe24-2b6d-4bb5-a30e-941636d0371f None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:31.260146 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1fe24-2b6d-4bb5-a30e-941636d0371f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:31.260517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1fe24-2b6d-4bb5-a30e-941636d0371f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:31.261944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1fe24-2b6d-4bb5-a30e-941636d0371f None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:31.273426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 319/641] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:31.274201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1fe24-2b6d-4bb5-a30e-941636d0371f None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:31.311030 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62438bdd-252b-4488-8c50-f68e0a1a119e None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:31.343795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22e8ba-2c3a-46e3-ab6c-2cbf3f023aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:31.344405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22e8ba-2c3a-46e3-ab6c-2cbf3f023aaf None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:31.344710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22e8ba-2c3a-46e3-ab6c-2cbf3f023aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:31.344981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22e8ba-2c3a-46e3-ab6c-2cbf3f023aaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:31.351869 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22e8ba-2c3a-46e3-ab6c-2cbf3f023aaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:31.353337 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22e8ba-2c3a-46e3-ab6c-2cbf3f023aaf None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:31.364544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-cf22e8ba-2c3a-46e3-ab6c-2cbf3f023aaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:31.364544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:31.364544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:31.364544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:31.364544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:31.364544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:31.364544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:31.364544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:31.364544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:31.364544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:31.364544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:31.364544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:31.364544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:31.364544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:31.364544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:31.365439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:31.366398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:31.366398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:31.366398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:31.366398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:31.366398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:31.366398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:31.366398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:31.366398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:31.366398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 323/642] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:31.366398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22e8ba-2c3a-46e3-ab6c-2cbf3f023aaf None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:31.375217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1fe24-2b6d-4bb5-a30e-941636d0371f None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:31.398850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:31.412734 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ce91d-3275-4cf7-9839-863c0c543583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:31.412734 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ce91d-3275-4cf7-9839-863c0c543583 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:31.412734 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ce91d-3275-4cf7-9839-863c0c543583 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:31.412734 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ce91d-3275-4cf7-9839-863c0c543583 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:31.413632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ce91d-3275-4cf7-9839-863c0c543583 None admin] RBAC: Adding query filter params (name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:31.413632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ce91d-3275-4cf7-9839-863c0c543583 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:31.413966 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ce91d-3275-4cf7-9839-863c0c543583 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:31.421629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 320/643] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:31.422404 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ce91d-3275-4cf7-9839-863c0c543583 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:31.455668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:31.464353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22e8ba-2c3a-46e3-ab6c-2cbf3f023aaf None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:31.498582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb08765-6f86-4d94-af42-8c102916af5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:31.499185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb08765-6f86-4d94-af42-8c102916af5b None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:31.499468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb08765-6f86-4d94-af42-8c102916af5b None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:31.499741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb08765-6f86-4d94-af42-8c102916af5b None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:31.500403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb08765-6f86-4d94-af42-8c102916af5b None admin] RBAC: Adding query filter params (name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:31.500739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb08765-6f86-4d94-af42-8c102916af5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:31.502146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb08765-6f86-4d94-af42-8c102916af5b None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:31.510007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 324/644] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:31.510793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb08765-6f86-4d94-af42-8c102916af5b None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:31.529813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ce91d-3275-4cf7-9839-863c0c543583 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:31.562919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ffbcf-19c6-47d5-a8a3-c0ff3247dbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:31.563515 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ffbcf-19c6-47d5-a8a3-c0ff3247dbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:31.563796 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ffbcf-19c6-47d5-a8a3-c0ff3247dbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:31.564067 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ffbcf-19c6-47d5-a8a3-c0ff3247dbf9 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:31.564783 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ffbcf-19c6-47d5-a8a3-c0ff3247dbf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:31.565088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ffbcf-19c6-47d5-a8a3-c0ff3247dbf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:31.566537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ffbcf-19c6-47d5-a8a3-c0ff3247dbf9 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:31.578169 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 321/645] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:31.578943 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ffbcf-19c6-47d5-a8a3-c0ff3247dbf9 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:31.611803 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb08765-6f86-4d94-af42-8c102916af5b None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:31.623473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:31.645568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915580d-389b-463e-a9d7-747c5a97f4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:31.646197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915580d-389b-463e-a9d7-747c5a97f4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:31.646474 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915580d-389b-463e-a9d7-747c5a97f4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:31.646747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915580d-389b-463e-a9d7-747c5a97f4c1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:31.654032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915580d-389b-463e-a9d7-747c5a97f4c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:31.655465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915580d-389b-463e-a9d7-747c5a97f4c1 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-a915580d-389b-463e-a9d7-747c5a97f4c1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:31.666441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:31.667414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:31.668329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:31.668329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:31.668329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:31.668329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:31.668329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:31.668329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:31.668329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:31.668329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:31.668329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 12 12:01:31.668329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:31.668329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 325/646] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:31.668329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915580d-389b-463e-a9d7-747c5a97f4c1 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:31.682337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ffbcf-19c6-47d5-a8a3-c0ff3247dbf9 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:31.683805 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:31.719751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157447cd-f3b9-4a43-81c7-8ed26d4021c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:31.720351 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157447cd-f3b9-4a43-81c7-8ed26d4021c0 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:31.720617 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157447cd-f3b9-4a43-81c7-8ed26d4021c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:31.720913 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157447cd-f3b9-4a43-81c7-8ed26d4021c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:31.721597 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157447cd-f3b9-4a43-81c7-8ed26d4021c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:31.722551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157447cd-f3b9-4a43-81c7-8ed26d4021c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:31.723960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157447cd-f3b9-4a43-81c7-8ed26d4021c0 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:31.734987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 322/647] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:31.735762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157447cd-f3b9-4a43-81c7-8ed26d4021c0 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:31.767530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915580d-389b-463e-a9d7-747c5a97f4c1 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:31.800613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba31346-52bf-4afb-9228-048c56781cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:31.801355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba31346-52bf-4afb-9228-048c56781cff None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:31.801640 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba31346-52bf-4afb-9228-048c56781cff None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:31.801911 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba31346-52bf-4afb-9228-048c56781cff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:31.815465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba31346-52bf-4afb-9228-048c56781cff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:31.817081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba31346-52bf-4afb-9228-048c56781cff None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-bba31346-52bf-4afb-9228-048c56781cff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:31.830833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:31.832612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:31.833687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:31.833687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:31.833687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:31.833687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 326/648] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:31.833944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157447cd-f3b9-4a43-81c7-8ed26d4021c0 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:31.834024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba31346-52bf-4afb-9228-048c56781cff None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:31.838636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:31.871248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f51824-c859-4739-b37c-9ce04e16d871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:31.871990 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f51824-c859-4739-b37c-9ce04e16d871 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:31.872274 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f51824-c859-4739-b37c-9ce04e16d871 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:31.872546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f51824-c859-4739-b37c-9ce04e16d871 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:31.883508 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f51824-c859-4739-b37c-9ce04e16d871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:31.884986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f51824-c859-4739-b37c-9ce04e16d871 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-14f51824-c859-4739-b37c-9ce04e16d871 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:31.899032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:31.900011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:31.900923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:31.900923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:31.900923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:31.900923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 323/649] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:31.900923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f51824-c859-4739-b37c-9ce04e16d871 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:31.902059 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:31.941031 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba31346-52bf-4afb-9228-048c56781cff None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:31.974911 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a60bf-b152-478d-9568-403f2f7241a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:31.975511 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a60bf-b152-478d-9568-403f2f7241a8 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:31.975789 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a60bf-b152-478d-9568-403f2f7241a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:31.976059 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a60bf-b152-478d-9568-403f2f7241a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:31.976811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a60bf-b152-478d-9568-403f2f7241a8 None admin] RBAC: Adding query filter params (role.id=1416a05bf5ba4de88889c54ff5c58da2, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=687d0375abd44d2481f988c6df90fad1) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:31.977122 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a60bf-b152-478d-9568-403f2f7241a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:31.978565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a60bf-b152-478d-9568-403f2f7241a8 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:31.984160 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 327/650] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:31 2022] GET /identity/v3/role_assignments?user.id=687d0375abd44d2481f988c6df90fad1&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=1416a05bf5ba4de88889c54ff5c58da2 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:31.984950 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a60bf-b152-478d-9568-403f2f7241a8 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:32.004686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f51824-c859-4739-b37c-9ce04e16d871 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:32.047108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0741e-0b03-4ad2-82e0-eccb92147eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:32.047794 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0741e-0b03-4ad2-82e0-eccb92147eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:32.048082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0741e-0b03-4ad2-82e0-eccb92147eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:32.048363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0741e-0b03-4ad2-82e0-eccb92147eb7 None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:32.049148 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0741e-0b03-4ad2-82e0-eccb92147eb7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:32.049469 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0741e-0b03-4ad2-82e0-eccb92147eb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:32.051675 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0741e-0b03-4ad2-82e0-eccb92147eb7 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:32.071203 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 324/651] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Aug 12 12:01:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:32.072068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0741e-0b03-4ad2-82e0-eccb92147eb7 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:32.091480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:32.104526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a60bf-b152-478d-9568-403f2f7241a8 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:32.140916 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:32.146999 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eeddc4-47d2-4b8e-ad7c-4a801514e9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:32.147662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eeddc4-47d2-4b8e-ad7c-4a801514e9cb None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:32.147962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eeddc4-47d2-4b8e-ad7c-4a801514e9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:32.148220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eeddc4-47d2-4b8e-ad7c-4a801514e9cb None admin] PATH_INFO: `/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/5123401076a84ffcab49b00fee65d14d/roles/1416a05bf5ba4de88889c54ff5c58da2` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:32.165941 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eeddc4-47d2-4b8e-ad7c-4a801514e9cb None admin] RBAC: Authorizing `identity:create_grant(project_id=20c5570631a34c4ab3868eb1cd96e345, user_id=5123401076a84ffcab49b00fee65d14d, role_id=1416a05bf5ba4de88889c54ff5c58da2)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:32.167499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eeddc4-47d2-4b8e-ad7c-4a801514e9cb None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:32.190268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0741e-0b03-4ad2-82e0-eccb92147eb7 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:32.227269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d312d4e-5680-47c5-a876-ca71253fe28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:32.227859 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d312d4e-5680-47c5-a876-ca71253fe28d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:32.228143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d312d4e-5680-47c5-a876-ca71253fe28d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:32.228420 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d312d4e-5680-47c5-a876-ca71253fe28d None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:32.229101 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d312d4e-5680-47c5-a876-ca71253fe28d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:32.229406 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d312d4e-5680-47c5-a876-ca71253fe28d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:32.230791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d312d4e-5680-47c5-a876-ca71253fe28d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:32.235421 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 328/652] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:01:31 2022] PUT /identity/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/5123401076a84ffcab49b00fee65d14d/roles/1416a05bf5ba4de88889c54ff5c58da2 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:01:32.236259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eeddc4-47d2-4b8e-ad7c-4a801514e9cb None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:32.246582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 325/653] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:32.247940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d312d4e-5680-47c5-a876-ca71253fe28d None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:32.347065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:32.379503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d312d4e-5680-47c5-a876-ca71253fe28d None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:32.399998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eeddc4-47d2-4b8e-ad7c-4a801514e9cb None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:32.423591 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16602ebe-0cef-4bf2-b6b4-af8e5dbc9fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:32.425380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16602ebe-0cef-4bf2-b6b4-af8e5dbc9fec None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:32.425826 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16602ebe-0cef-4bf2-b6b4-af8e5dbc9fec None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:32.426246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16602ebe-0cef-4bf2-b6b4-af8e5dbc9fec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:32.440590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a890eb-87b9-4820-8264-defd85169780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:32.442152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a890eb-87b9-4820-8264-defd85169780 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:32.442554 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a890eb-87b9-4820-8264-defd85169780 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:32.442877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a890eb-87b9-4820-8264-defd85169780 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:32.443612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a890eb-87b9-4820-8264-defd85169780 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:32.443990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a890eb-87b9-4820-8264-defd85169780 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:32.445531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a890eb-87b9-4820-8264-defd85169780 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:32.458007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 329/654] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:32.458998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a890eb-87b9-4820-8264-defd85169780 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:32.503605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:32.518363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16602ebe-0cef-4bf2-b6b4-af8e5dbc9fec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:32.520911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16602ebe-0cef-4bf2-b6b4-af8e5dbc9fec None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:32.572679 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a890eb-87b9-4820-8264-defd85169780 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:32.607504 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e5008-ed81-4fbf-b7fa-c1b8cb35d980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:32.608140 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e5008-ed81-4fbf-b7fa-c1b8cb35d980 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:32.608424 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e5008-ed81-4fbf-b7fa-c1b8cb35d980 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:32.608726 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e5008-ed81-4fbf-b7fa-c1b8cb35d980 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:32.619047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e5008-ed81-4fbf-b7fa-c1b8cb35d980 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:32.620507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e5008-ed81-4fbf-b7fa-c1b8cb35d980 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-9d9e5008-ed81-4fbf-b7fa-c1b8cb35d980 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:32.632971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:32.633921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:32.634807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:32.634807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:32.634807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:32.634807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:32.634807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:32.634807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:32.634807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:32.634807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:32.634807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 12:01:32.634807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:32.634807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 330/655] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:32.635385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e5008-ed81-4fbf-b7fa-c1b8cb35d980 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:32.646217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 326/656] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:32.647438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16602ebe-0cef-4bf2-b6b4-af8e5dbc9fec None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:32.652987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:32.748803 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:32.751926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e5008-ed81-4fbf-b7fa-c1b8cb35d980 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:32.756822 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16602ebe-0cef-4bf2-b6b4-af8e5dbc9fec None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:32.790432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51b14d-3889-4f6a-bed0-4556c97875be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:32.791058 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51b14d-3889-4f6a-bed0-4556c97875be None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:32.791341 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51b14d-3889-4f6a-bed0-4556c97875be None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:32.791616 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51b14d-3889-4f6a-bed0-4556c97875be None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:32.794655 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d97262-c828-47a3-ab30-39ea9d6e1f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:32.795468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d97262-c828-47a3-ab30-39ea9d6e1f2c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:32.795943 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d97262-c828-47a3-ab30-39ea9d6e1f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:32.796429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d97262-c828-47a3-ab30-39ea9d6e1f2c None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:32.797397 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d97262-c828-47a3-ab30-39ea9d6e1f2c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:32.797899 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d97262-c828-47a3-ab30-39ea9d6e1f2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:32.799323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51b14d-3889-4f6a-bed0-4556c97875be None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:32.799715 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d97262-c828-47a3-ab30-39ea9d6e1f2c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:32.801158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51b14d-3889-4f6a-bed0-4556c97875be None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-cc51b14d-3889-4f6a-bed0-4556c97875be None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:32.814480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:32.815374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:32.816982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:32.816982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:32.816982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:32.816982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:32.816982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:32.816982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:32.816982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:32.816982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:32.816982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 12 12:01:32.816982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:32.820909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 331/657] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:32.822326 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51b14d-3889-4f6a-bed0-4556c97875be None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:32.826281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 327/658] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Aug 12 12:01:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:32.827366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d97262-c828-47a3-ab30-39ea9d6e1f2c None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:32.860169 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:32.933634 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d97262-c828-47a3-ab30-39ea9d6e1f2c None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:32.935171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51b14d-3889-4f6a-bed0-4556c97875be None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:32.970924 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585acc2f-450f-4510-b238-ff943926317a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:32.972259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d8436-3c3b-41d3-8042-16f6b4235b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:32.972667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585acc2f-450f-4510-b238-ff943926317a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:32.972963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d8436-3c3b-41d3-8042-16f6b4235b8e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:32.973261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d8436-3c3b-41d3-8042-16f6b4235b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:32.973487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d8436-3c3b-41d3-8042-16f6b4235b8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:32.973854 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585acc2f-450f-4510-b238-ff943926317a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:32.974312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585acc2f-450f-4510-b238-ff943926317a None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:32.975341 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585acc2f-450f-4510-b238-ff943926317a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:32.975839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585acc2f-450f-4510-b238-ff943926317a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:32.977582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585acc2f-450f-4510-b238-ff943926317a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:32.984569 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d8436-3c3b-41d3-8042-16f6b4235b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:32.986162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d8436-3c3b-41d3-8042-16f6b4235b8e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:32.997777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 328/659] 10.222.0.27 () {56 vars in 1139 bytes} [Fri Aug 12 12:01:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:32.998853 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585acc2f-450f-4510-b238-ff943926317a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-c76d8436-3c3b-41d3-8042-16f6b4235b8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:33.005806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:33.006730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:33.007632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:33.007632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:33.007632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:33.007632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 332/660] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:33.008276 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d8436-3c3b-41d3-8042-16f6b4235b8e None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:33.024943 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:33.099050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:33.111499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585acc2f-450f-4510-b238-ff943926317a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:33.120371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d8436-3c3b-41d3-8042-16f6b4235b8e None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:33.158281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457c8b8-5500-450e-b040-2f809873b71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:33.158889 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457c8b8-5500-450e-b040-2f809873b71b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:33.159366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457c8b8-5500-450e-b040-2f809873b71b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:33.159650 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457c8b8-5500-450e-b040-2f809873b71b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:33.159970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed9b01-ef82-41b8-af8f-1db2885f7752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:33.160580 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed9b01-ef82-41b8-af8f-1db2885f7752 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:33.160890 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed9b01-ef82-41b8-af8f-1db2885f7752 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:33.161307 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed9b01-ef82-41b8-af8f-1db2885f7752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:33.235162 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457c8b8-5500-450e-b040-2f809873b71b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:33.236827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457c8b8-5500-450e-b040-2f809873b71b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:33.243665 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed9b01-ef82-41b8-af8f-1db2885f7752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:33.245274 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed9b01-ef82-41b8-af8f-1db2885f7752 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:33.260161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:33.332944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:33.343781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 329/661] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:33.344776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457c8b8-5500-450e-b040-2f809873b71b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:33.367137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 333/662] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:33.368084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed9b01-ef82-41b8-af8f-1db2885f7752 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:33.453496 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457c8b8-5500-450e-b040-2f809873b71b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:33.478581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed9b01-ef82-41b8-af8f-1db2885f7752 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:33.490571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb13e2-afaf-462a-9b82-122711672826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:33.491256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb13e2-afaf-462a-9b82-122711672826 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:33.491612 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb13e2-afaf-462a-9b82-122711672826 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:33.491950 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb13e2-afaf-462a-9b82-122711672826 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:33.492872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:33.493328 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb13e2-afaf-462a-9b82-122711672826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:33.493638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb13e2-afaf-462a-9b82-122711672826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:33.495002 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb13e2-afaf-462a-9b82-122711672826 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:33.507359 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 330/663] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:33.508259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb13e2-afaf-462a-9b82-122711672826 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:33.520465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7782160-e53f-4869-99af-d92b638ae689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:33.521416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7782160-e53f-4869-99af-d92b638ae689 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:33.521968 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7782160-e53f-4869-99af-d92b638ae689 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:33.522435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7782160-e53f-4869-99af-d92b638ae689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:33.533957 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7782160-e53f-4869-99af-d92b638ae689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:33.536145 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7782160-e53f-4869-99af-d92b638ae689 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-d7782160-e53f-4869-99af-d92b638ae689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:33.551407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:33.552312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:33.553652 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:33.553652 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:33.553652 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:33.555685 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 334/664] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:33.556758 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7782160-e53f-4869-99af-d92b638ae689 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:33.561785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:33.620266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb13e2-afaf-462a-9b82-122711672826 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:33.664499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddbb2f-1c25-4639-9a64-6a701db23b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:33.665603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddbb2f-1c25-4639-9a64-6a701db23b81 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:33.666078 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddbb2f-1c25-4639-9a64-6a701db23b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:33.666572 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddbb2f-1c25-4639-9a64-6a701db23b81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:33.670856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7782160-e53f-4869-99af-d92b638ae689 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:33.678976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddbb2f-1c25-4639-9a64-6a701db23b81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:33.681188 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddbb2f-1c25-4639-9a64-6a701db23b81 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-87ddbb2f-1c25-4639-9a64-6a701db23b81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:33.698071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:33.699070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:33.699984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:33.699984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:33.699984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:33.702265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 331/665] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:33.703294 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddbb2f-1c25-4639-9a64-6a701db23b81 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:33.707714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec5b33-1356-4326-a8d2-0ad49f99600d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:33.708426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec5b33-1356-4326-a8d2-0ad49f99600d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:33.708731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec5b33-1356-4326-a8d2-0ad49f99600d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:33.709606 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec5b33-1356-4326-a8d2-0ad49f99600d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:33.716238 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec5b33-1356-4326-a8d2-0ad49f99600d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:33.717716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec5b33-1356-4326-a8d2-0ad49f99600d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:33.729981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-18ec5b33-1356-4326-a8d2-0ad49f99600d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:33.729981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:33.729981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:33.729981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:33.729981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:33.729981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:33.729981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:33.729981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:33.729981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:33.729981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:33.729981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:33.729981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:33.729981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:33.729981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:33.729981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:33.730891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:33.731749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:33.731749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:33.731749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:33.731749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:33.731749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:33.731749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:33.731749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:33.731749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:33.731749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 335/666] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:33.732689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec5b33-1356-4326-a8d2-0ad49f99600d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:33.758809 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:33.824328 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddbb2f-1c25-4639-9a64-6a701db23b81 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:33.826232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:33.838868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec5b33-1356-4326-a8d2-0ad49f99600d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:33.864284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fce8b-01f7-40f3-a05a-2b1e87176357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:33.865270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fce8b-01f7-40f3-a05a-2b1e87176357 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:33.865748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fce8b-01f7-40f3-a05a-2b1e87176357 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:33.866213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fce8b-01f7-40f3-a05a-2b1e87176357 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:33.867642 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fce8b-01f7-40f3-a05a-2b1e87176357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:33.868151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fce8b-01f7-40f3-a05a-2b1e87176357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:33.869868 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fce8b-01f7-40f3-a05a-2b1e87176357 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:33.875130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402b16e-2724-4639-8efe-d5e9b62d026a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:33.875738 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402b16e-2724-4639-8efe-d5e9b62d026a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:33.876020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402b16e-2724-4639-8efe-d5e9b62d026a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:33.876293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402b16e-2724-4639-8efe-d5e9b62d026a None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:33.876966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402b16e-2724-4639-8efe-d5e9b62d026a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:33.877328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402b16e-2724-4639-8efe-d5e9b62d026a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:33.878958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402b16e-2724-4639-8efe-d5e9b62d026a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:33.884935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 332/667] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:33.885995 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fce8b-01f7-40f3-a05a-2b1e87176357 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:33.892060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 336/668] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:33.893077 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402b16e-2724-4639-8efe-d5e9b62d026a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:33.986314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:34.002908 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402b16e-2724-4639-8efe-d5e9b62d026a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:34.004863 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fce8b-01f7-40f3-a05a-2b1e87176357 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:34.039905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdd1b6-16b1-45e5-a75c-0fcdd3e9e8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:34.040562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdd1b6-16b1-45e5-a75c-0fcdd3e9e8fe None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:34.040843 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdd1b6-16b1-45e5-a75c-0fcdd3e9e8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:34.041116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdd1b6-16b1-45e5-a75c-0fcdd3e9e8fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:34.041864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5255b-a104-41be-b700-3c805a04399c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:34.042679 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5255b-a104-41be-b700-3c805a04399c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:34.043156 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5255b-a104-41be-b700-3c805a04399c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:34.043624 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5255b-a104-41be-b700-3c805a04399c None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:34.044520 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5255b-a104-41be-b700-3c805a04399c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:34.045043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5255b-a104-41be-b700-3c805a04399c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:34.046757 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5255b-a104-41be-b700-3c805a04399c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:34.048230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdd1b6-16b1-45e5-a75c-0fcdd3e9e8fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:34.050382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdd1b6-16b1-45e5-a75c-0fcdd3e9e8fe None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:34.060835 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 333/669] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:34.062052 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5255b-a104-41be-b700-3c805a04399c None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:34.062536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-f7fdd1b6-16b1-45e5-a75c-0fcdd3e9e8fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:34.062536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:34.062536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:34.062536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:34.062536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:34.062536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:34.062536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:34.062536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:34.062536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:34.062536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:34.062536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:34.062536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:34.062536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:34.062536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:34.062536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:34.063435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:34.064436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:34.064436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:34.064436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:34.064436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:34.064436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:34.064436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:34.064436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:34.064436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:34.064436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 337/670] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:34.066289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdd1b6-16b1-45e5-a75c-0fcdd3e9e8fe None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:34.090867 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:34.167608 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdd1b6-16b1-45e5-a75c-0fcdd3e9e8fe None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:34.181041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5255b-a104-41be-b700-3c805a04399c None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:34.203985 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28c72a-2fc6-482b-8d78-10bedc6522ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:34.204621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28c72a-2fc6-482b-8d78-10bedc6522ac None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:34.204945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28c72a-2fc6-482b-8d78-10bedc6522ac None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:34.205277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28c72a-2fc6-482b-8d78-10bedc6522ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:34.206003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28c72a-2fc6-482b-8d78-10bedc6522ac None admin] RBAC: Adding query filter params (role.id=1416a05bf5ba4de88889c54ff5c58da2, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=ffe9f3fec7104a289e05e88e2d751458) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:34.206491 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28c72a-2fc6-482b-8d78-10bedc6522ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:34.207927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28c72a-2fc6-482b-8d78-10bedc6522ac None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:34.225075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 338/671] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:34 2022] GET /identity/v3/role_assignments?user.id=ffe9f3fec7104a289e05e88e2d751458&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=1416a05bf5ba4de88889c54ff5c58da2 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:34.225075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28c72a-2fc6-482b-8d78-10bedc6522ac None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:34.225909 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63558305-e306-438f-91a6-27ad5a886ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:34.226570 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63558305-e306-438f-91a6-27ad5a886ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:34.226917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63558305-e306-438f-91a6-27ad5a886ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:34.227201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63558305-e306-438f-91a6-27ad5a886ed4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:34.235621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63558305-e306-438f-91a6-27ad5a886ed4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:34.239965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63558305-e306-438f-91a6-27ad5a886ed4 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:34.240447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:34.251680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-63558305-e306-438f-91a6-27ad5a886ed4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:34.251680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:34.251680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:34.251680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:34.251680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:34.251680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:34.251680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:34.251680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:34.251680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:34.251680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:34.251680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:34.251680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:34.251680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:34.251680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:34.251680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:34.252627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:34.253596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:34.253596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:34.253596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:34.253596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:34.253596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:34.253596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:34.253596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:34.253596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:34.253596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 334/672] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:34.255241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6420c-50cc-4d90-8ce1-ab5ec6288fb5 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:34.255241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6420c-50cc-4d90-8ce1-ab5ec6288fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:34.255241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6420c-50cc-4d90-8ce1-ab5ec6288fb5 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:34.260665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 335/673] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:34.260665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6420c-50cc-4d90-8ce1-ab5ec6288fb5 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:34.356302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28c72a-2fc6-482b-8d78-10bedc6522ac None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:34.372353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:34.396753 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6420c-50cc-4d90-8ce1-ab5ec6288fb5 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:34.398421 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebef0f-5d3a-4c5c-89a0-8517eac82625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:34.399881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebef0f-5d3a-4c5c-89a0-8517eac82625 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:34.399881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebef0f-5d3a-4c5c-89a0-8517eac82625 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:34.399881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebef0f-5d3a-4c5c-89a0-8517eac82625 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:34.400266 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebef0f-5d3a-4c5c-89a0-8517eac82625 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:34.400564 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebef0f-5d3a-4c5c-89a0-8517eac82625 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:34.402527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebef0f-5d3a-4c5c-89a0-8517eac82625 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:34.415084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 339/674] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:34.416237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fccc7b-d0a7-4b36-9e76-85c8fff7c9b6 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:34.416519 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fccc7b-d0a7-4b36-9e76-85c8fff7c9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:34.416822 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fccc7b-d0a7-4b36-9e76-85c8fff7c9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:34.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f2fccc7b-d0a7-4b36-9e76-85c8fff7c9b6 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:34.437874 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f2fccc7b-d0a7-4b36-9e76-85c8fff7c9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:34.441056 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f3745-3476-4627-865e-99cc3f25c889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:34.441540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f3745-3476-4627-865e-99cc3f25c889 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:34.441822 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f3745-3476-4627-865e-99cc3f25c889 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:34.442093 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f3745-3476-4627-865e-99cc3f25c889 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:34.442965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f3745-3476-4627-865e-99cc3f25c889 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:34.443143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f3745-3476-4627-865e-99cc3f25c889 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:34.444552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f3745-3476-4627-865e-99cc3f25c889 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:34.458164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 336/675] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:34.459400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca5a61-36a6-4b18-9cc4-bc4ade9bf896 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:34.459689 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca5a61-36a6-4b18-9cc4-bc4ade9bf896 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:34.459963 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca5a61-36a6-4b18-9cc4-bc4ade9bf896 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:34.461489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 337/676] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:34.462147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca5a61-36a6-4b18-9cc4-bc4ade9bf896 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:34.511674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:34.517784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 340/677] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:34.519660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fccc7b-d0a7-4b36-9e76-85c8fff7c9b6 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:34.572426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca5a61-36a6-4b18-9cc4-bc4ade9bf896 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:34.611469 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873b77e-1863-4250-bf15-e250d3ec7f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:34.611987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873b77e-1863-4250-bf15-e250d3ec7f5c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:34.612825 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873b77e-1863-4250-bf15-e250d3ec7f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:34.613100 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873b77e-1863-4250-bf15-e250d3ec7f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:34.613888 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873b77e-1863-4250-bf15-e250d3ec7f5c None admin] RBAC: Adding query filter params (role.id=1416a05bf5ba4de88889c54ff5c58da2, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=935ab3a263ae46fcb929029a147a1010) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:34.614193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873b77e-1863-4250-bf15-e250d3ec7f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:34.615679 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873b77e-1863-4250-bf15-e250d3ec7f5c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:34.622309 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 338/678] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:34 2022] GET /identity/v3/role_assignments?user.id=935ab3a263ae46fcb929029a147a1010&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=1416a05bf5ba4de88889c54ff5c58da2 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:34.623629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16c563-1ab6-423d-b2af-bfa889d17a53 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:34.623913 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16c563-1ab6-423d-b2af-bfa889d17a53 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:34.624178 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16c563-1ab6-423d-b2af-bfa889d17a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:34.647157 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:34.647967 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fccc7b-d0a7-4b36-9e76-85c8fff7c9b6 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:34.651425 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-4c16c563-1ab6-423d-b2af-bfa889d17a53 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:34.652683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4c16c563-1ab6-423d-b2af-bfa889d17a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:34.701994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d874b63-9a29-44f7-b095-2888b56cc6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:34.702625 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d874b63-9a29-44f7-b095-2888b56cc6d8 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:34.702909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d874b63-9a29-44f7-b095-2888b56cc6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:34.703187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d874b63-9a29-44f7-b095-2888b56cc6d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:34.703949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d874b63-9a29-44f7-b095-2888b56cc6d8 None admin] RBAC: Adding query filter params (role.id=1416a05bf5ba4de88889c54ff5c58da2, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=745fad9c109a4280906eb0349849117d) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:34.704258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d874b63-9a29-44f7-b095-2888b56cc6d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:34.705766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d874b63-9a29-44f7-b095-2888b56cc6d8 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:34.721700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 341/679] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:34 2022] GET /identity/v3/role_assignments?user.id=745fad9c109a4280906eb0349849117d&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=1416a05bf5ba4de88889c54ff5c58da2 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:34.789677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 339/680] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:34.864622 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d874b63-9a29-44f7-b095-2888b56cc6d8 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:34.933508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:34.977958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d874b63-9a29-44f7-b095-2888b56cc6d8 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:35.017102 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f0a1c-4110-46c8-9459-9425b337add4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:35.018079 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f0a1c-4110-46c8-9459-9425b337add4 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:35.019337 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f0a1c-4110-46c8-9459-9425b337add4 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:35.020166 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f0a1c-4110-46c8-9459-9425b337add4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:35.032439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f0a1c-4110-46c8-9459-9425b337add4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-421f0a1c-4110-46c8-9459-9425b337add4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:35.038740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:35.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:35.040563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:35.040563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:35.040563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:35.040563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:35.040563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:35.040563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:35.040563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:35.040563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:35.040563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:35.040563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:35.040563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:35.040563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:35.047687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 342/681] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:01:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:35.049293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16c563-1ab6-423d-b2af-bfa889d17a53 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:35.111019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f0a1c-4110-46c8-9459-9425b337add4 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:35.201440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16c563-1ab6-423d-b2af-bfa889d17a53 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:35.272622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27920b8-756b-4098-9622-d9d2886da896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:35.273254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27920b8-756b-4098-9622-d9d2886da896 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:35.273534 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27920b8-756b-4098-9622-d9d2886da896 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:35.273864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27920b8-756b-4098-9622-d9d2886da896 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:35.274540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27920b8-756b-4098-9622-d9d2886da896 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:35.274841 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27920b8-756b-4098-9622-d9d2886da896 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:35.276221 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27920b8-756b-4098-9622-d9d2886da896 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:35.289442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 340/682] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:35.292987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:35.295941 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f0a1c-4110-46c8-9459-9425b337add4 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:35.305087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27920b8-756b-4098-9622-d9d2886da896 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:35.337696 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:35.360177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eec2f1-8cd6-47f4-9ded-5bee02c4b2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:35.360847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eec2f1-8cd6-47f4-9ded-5bee02c4b2ef None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:35.361127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eec2f1-8cd6-47f4-9ded-5bee02c4b2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:35.361400 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eec2f1-8cd6-47f4-9ded-5bee02c4b2ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:35.387260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eec2f1-8cd6-47f4-9ded-5bee02c4b2ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-13eec2f1-8cd6-47f4-9ded-5bee02c4b2ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:35.391984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:35.393023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:35.394452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:35.394452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:35.394452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:35.394452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:35.394452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:35.394452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:35.394452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:35.394452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:35.394452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:35.394452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:35.394452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:35.394452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:35.394452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 343/683] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:01:35.399833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eec2f1-8cd6-47f4-9ded-5bee02c4b2ef None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:35.485598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27920b8-756b-4098-9622-d9d2886da896 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:35.513690 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eec2f1-8cd6-47f4-9ded-5bee02c4b2ef None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:35.557519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15877ada-aa33-40a7-a47c-8d472cdf7b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:35.558713 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15877ada-aa33-40a7-a47c-8d472cdf7b81 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:35.560144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15877ada-aa33-40a7-a47c-8d472cdf7b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:35.560633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15877ada-aa33-40a7-a47c-8d472cdf7b81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:35.574444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14512e5-78dc-4890-afa0-1135157c9632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:35.575381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14512e5-78dc-4890-afa0-1135157c9632 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:35.575860 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14512e5-78dc-4890-afa0-1135157c9632 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:35.576422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14512e5-78dc-4890-afa0-1135157c9632 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:35.577312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14512e5-78dc-4890-afa0-1135157c9632 None admin] RBAC: Adding query filter params (name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:35.577948 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14512e5-78dc-4890-afa0-1135157c9632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:35.581538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14512e5-78dc-4890-afa0-1135157c9632 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:35.584911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15877ada-aa33-40a7-a47c-8d472cdf7b81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:35.587560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15877ada-aa33-40a7-a47c-8d472cdf7b81 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:35.602087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 344/684] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:35.606856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14512e5-78dc-4890-afa0-1135157c9632 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-15877ada-aa33-40a7-a47c-8d472cdf7b81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:35.619514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:35.620488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:35.625611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:35.625611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:35.625611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:35.625611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 341/685] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:01:35.641649 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15877ada-aa33-40a7-a47c-8d472cdf7b81 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:35.666125 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:35.726465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14512e5-78dc-4890-afa0-1135157c9632 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:35.740274 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:35.764212 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f8261-2808-4a36-a419-c19e849a1b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:35.765289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f8261-2808-4a36-a419-c19e849a1b41 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:35.765793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f8261-2808-4a36-a419-c19e849a1b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:35.766265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f8261-2808-4a36-a419-c19e849a1b41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:35.777710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f8261-2808-4a36-a419-c19e849a1b41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:35.779515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f8261-2808-4a36-a419-c19e849a1b41 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:35.793670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15877ada-aa33-40a7-a47c-8d472cdf7b81 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-244f8261-2808-4a36-a419-c19e849a1b41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:35.812509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:35.813496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:35.814970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:35.814970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:35.814970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:35.814970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 345/686] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:35.820093 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f8261-2808-4a36-a419-c19e849a1b41 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:35.856852 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac04591-5417-4a53-9b91-4da44432db28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:35.857457 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac04591-5417-4a53-9b91-4da44432db28 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:35.857736 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac04591-5417-4a53-9b91-4da44432db28 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:35.858010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac04591-5417-4a53-9b91-4da44432db28 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:35.858657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac04591-5417-4a53-9b91-4da44432db28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:35.858955 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac04591-5417-4a53-9b91-4da44432db28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:35.860318 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac04591-5417-4a53-9b91-4da44432db28 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:35.888611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 342/687] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:35.898834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac04591-5417-4a53-9b91-4da44432db28 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:36.019288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:36.028878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f8261-2808-4a36-a419-c19e849a1b41 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:36.085233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac04591-5417-4a53-9b91-4da44432db28 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:36.102353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:36.102556 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bdee9-2e27-4bbb-8121-0baff333f4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:36.103048 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bdee9-2e27-4bbb-8121-0baff333f4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:36.103668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bdee9-2e27-4bbb-8121-0baff333f4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:36.103668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bdee9-2e27-4bbb-8121-0baff333f4d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:36.108185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bdee9-2e27-4bbb-8121-0baff333f4d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:36.108185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bdee9-2e27-4bbb-8121-0baff333f4d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:36.108185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bdee9-2e27-4bbb-8121-0baff333f4d7 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:36.127033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 346/688] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:36.132182 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bdee9-2e27-4bbb-8121-0baff333f4d7 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:36.152088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eccb92-82bf-4110-8472-fd2d7759c0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:36.153096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eccb92-82bf-4110-8472-fd2d7759c0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:36.153585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eccb92-82bf-4110-8472-fd2d7759c0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:36.154049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eccb92-82bf-4110-8472-fd2d7759c0d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:36.169666 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eccb92-82bf-4110-8472-fd2d7759c0d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:36.172332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eccb92-82bf-4110-8472-fd2d7759c0d0 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-27eccb92-82bf-4110-8472-fd2d7759c0d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:36.190892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:36.191915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:36.192872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:36.192872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:36.192872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:36.192872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:36.192872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:36.192872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:36.192872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:36.192872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:36.192872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:01:36.192872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:36.196057 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 343/689] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 12 12:01:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:36.201342 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eccb92-82bf-4110-8472-fd2d7759c0d0 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:36.314366 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bdee9-2e27-4bbb-8121-0baff333f4d7 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:36.363915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e5892-6391-4bbf-bdad-1acb9a3aa2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:36.364951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e5892-6391-4bbf-bdad-1acb9a3aa2fe None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:36.365428 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e5892-6391-4bbf-bdad-1acb9a3aa2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:36.365891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e5892-6391-4bbf-bdad-1acb9a3aa2fe None admin] PATH_INFO: `/v3/users/swift` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:36.375326 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e5892-6391-4bbf-bdad-1acb9a3aa2fe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:36.377710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e5892-6391-4bbf-bdad-1acb9a3aa2fe None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-445e5892-6391-4bbf-bdad-1acb9a3aa2fe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:36.393477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:36.394384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:36.395288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:36.395288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:36.395288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:36.395288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:36.395288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:36.395288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:36.395288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:36.395288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:36.395288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 12 12:01:36.395288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:36.397751 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 347/690] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:01:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:36.398639 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eccb92-82bf-4110-8472-fd2d7759c0d0 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:36.399244 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cade8ba-4da0-4a99-91df-b3c3397b1399 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:36.399665 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cade8ba-4da0-4a99-91df-b3c3397b1399 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:36.400087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cade8ba-4da0-4a99-91df-b3c3397b1399 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:36.402710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 348/691] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:36.403557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cade8ba-4da0-4a99-91df-b3c3397b1399 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:36.430238 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:36.446156 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc41e82-8c13-4455-beb0-7a175c42d18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:36.447210 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc41e82-8c13-4455-beb0-7a175c42d18c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:36.447676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc41e82-8c13-4455-beb0-7a175c42d18c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:36.448130 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc41e82-8c13-4455-beb0-7a175c42d18c None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:36.449224 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc41e82-8c13-4455-beb0-7a175c42d18c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:36.450482 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc41e82-8c13-4455-beb0-7a175c42d18c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:36.452076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc41e82-8c13-4455-beb0-7a175c42d18c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:36.473316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 344/692] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 12 12:01:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:36.474712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d02f3f-19e9-4d12-b254-307e8f278a01 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:36.474966 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d02f3f-19e9-4d12-b254-307e8f278a01 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:36.475313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d02f3f-19e9-4d12-b254-307e8f278a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:36.494066 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-c9d02f3f-19e9-4d12-b254-307e8f278a01 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:36.496181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c9d02f3f-19e9-4d12-b254-307e8f278a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:36.520584 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:36.534740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cade8ba-4da0-4a99-91df-b3c3397b1399 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:36.573019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 345/693] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:36.573985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d02f3f-19e9-4d12-b254-307e8f278a01 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:36.576878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5fffd-d7fc-478f-ad3e-5b8e9823f6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:36.577742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5fffd-d7fc-478f-ad3e-5b8e9823f6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:36.578180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5fffd-d7fc-478f-ad3e-5b8e9823f6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:36.578622 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5fffd-d7fc-478f-ad3e-5b8e9823f6c6 None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:36.579496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5fffd-d7fc-478f-ad3e-5b8e9823f6c6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:36.579967 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5fffd-d7fc-478f-ad3e-5b8e9823f6c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:36.582715 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5fffd-d7fc-478f-ad3e-5b8e9823f6c6 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:36.603274 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 349/694] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Aug 12 12:01:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:36.608804 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5fffd-d7fc-478f-ad3e-5b8e9823f6c6 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:36.688250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:36.715449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5fffd-d7fc-478f-ad3e-5b8e9823f6c6 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:36.752123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c5f57-834f-4b7d-80da-7f14b6ab528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:36.752981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c5f57-834f-4b7d-80da-7f14b6ab528f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:36.753282 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c5f57-834f-4b7d-80da-7f14b6ab528f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:36.753560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c5f57-834f-4b7d-80da-7f14b6ab528f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:36.786669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d02f3f-19e9-4d12-b254-307e8f278a01 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:36.835760 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c5f57-834f-4b7d-80da-7f14b6ab528f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:36.837568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c5f57-834f-4b7d-80da-7f14b6ab528f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:36.871890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3ec4d-4e47-4295-9749-e9de0b70ff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:36.872496 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3ec4d-4e47-4295-9749-e9de0b70ff7b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:36.872928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3ec4d-4e47-4295-9749-e9de0b70ff7b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:36.873085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3ec4d-4e47-4295-9749-e9de0b70ff7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:36.910440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:36.954048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3ec4d-4e47-4295-9749-e9de0b70ff7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:36.955607 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3ec4d-4e47-4295-9749-e9de0b70ff7b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:36.971747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 350/695] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:36.971747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:36.974578 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60212a0e-71ae-481d-85de-8561bd1fed81 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:36.974876 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60212a0e-71ae-481d-85de-8561bd1fed81 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:36.975146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60212a0e-71ae-481d-85de-8561bd1fed81 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:36.979178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 351/696] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:36.979178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60212a0e-71ae-481d-85de-8561bd1fed81 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:37.081060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 346/697] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 12 12:01:37.082313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec8e7d-7e71-4859-a640-bf83d5f7010a None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:37.082592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec8e7d-7e71-4859-a640-bf83d5f7010a None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:37.082863 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec8e7d-7e71-4859-a640-bf83d5f7010a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:37.092467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60212a0e-71ae-481d-85de-8561bd1fed81 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:37.102198 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-1dec8e7d-7e71-4859-a640-bf83d5f7010a None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:37.103696 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1dec8e7d-7e71-4859-a640-bf83d5f7010a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:37.129222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f6171-a379-4202-89b5-cd147ee575c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:37.129852 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f6171-a379-4202-89b5-cd147ee575c2 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:37.130301 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f6171-a379-4202-89b5-cd147ee575c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:37.130406 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f6171-a379-4202-89b5-cd147ee575c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:37.140979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f6171-a379-4202-89b5-cd147ee575c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:37.142493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f6171-a379-4202-89b5-cd147ee575c2 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-100f6171-a379-4202-89b5-cd147ee575c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:37.157039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:37.157966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:37.159362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:37.159362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:37.159362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:37.160549 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:37.163317 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 352/698] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:37.164029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f6171-a379-4202-89b5-cd147ee575c2 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:37.173838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 347/699] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:37.175213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec8e7d-7e71-4859-a640-bf83d5f7010a None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:37.199551 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:37.278524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec8e7d-7e71-4859-a640-bf83d5f7010a None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:37.282846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f6171-a379-4202-89b5-cd147ee575c2 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:37.317945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200b2d1-60a1-43e2-a491-445bd1b9d206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:37.318572 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200b2d1-60a1-43e2-a491-445bd1b9d206 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:37.318847 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200b2d1-60a1-43e2-a491-445bd1b9d206 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:37.319116 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200b2d1-60a1-43e2-a491-445bd1b9d206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:37.320467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360680e9-5503-411c-842f-5828313e4cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:37.321131 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360680e9-5503-411c-842f-5828313e4cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:37.321409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360680e9-5503-411c-842f-5828313e4cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:37.321681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360680e9-5503-411c-842f-5828313e4cc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:37.329681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200b2d1-60a1-43e2-a491-445bd1b9d206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:37.331701 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200b2d1-60a1-43e2-a491-445bd1b9d206 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:37.333021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360680e9-5503-411c-842f-5828313e4cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:37.335011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360680e9-5503-411c-842f-5828313e4cc3 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-e200b2d1-60a1-43e2-a491-445bd1b9d206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:37.346021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:37.347103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:37.348417 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:37.348417 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:37.348417 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-360680e9-5503-411c-842f-5828313e4cc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:37.349900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:37.350768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:37.352218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:37.352218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:37.352218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:37.352391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 348/700] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:37.353169 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c88690-46c4-4c2c-9134-adec142e4c0f None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:37.353454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c88690-46c4-4c2c-9134-adec142e4c0f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:37.353729 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c88690-46c4-4c2c-9134-adec142e4c0f None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:37.356701 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 349/701] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:37.356837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 353/702] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:37.356935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c88690-46c4-4c2c-9134-adec142e4c0f None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:37.357011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360680e9-5503-411c-842f-5828313e4cc3 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:37.410892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:37.444437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:37.465933 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360680e9-5503-411c-842f-5828313e4cc3 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:37.469260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c88690-46c4-4c2c-9134-adec142e4c0f None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:37.499963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad791aff-74f9-4c33-853d-a7b4dc5bbb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:37.500565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad791aff-74f9-4c33-853d-a7b4dc5bbb40 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:37.500875 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad791aff-74f9-4c33-853d-a7b4dc5bbb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:37.501151 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad791aff-74f9-4c33-853d-a7b4dc5bbb40 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:37.501789 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad791aff-74f9-4c33-853d-a7b4dc5bbb40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:37.502096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad791aff-74f9-4c33-853d-a7b4dc5bbb40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:37.502441 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523945c-d51e-4471-b625-5ab7d613ec02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:37.503797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523945c-d51e-4471-b625-5ab7d613ec02 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:37.504071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523945c-d51e-4471-b625-5ab7d613ec02 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:37.504170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad791aff-74f9-4c33-853d-a7b4dc5bbb40 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:37.504341 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523945c-d51e-4471-b625-5ab7d613ec02 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:37.504999 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523945c-d51e-4471-b625-5ab7d613ec02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:37.505307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523945c-d51e-4471-b625-5ab7d613ec02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:37.506645 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523945c-d51e-4471-b625-5ab7d613ec02 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:37.515755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 354/703] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:37.516550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad791aff-74f9-4c33-853d-a7b4dc5bbb40 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:37.518122 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 350/704] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:37.519139 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73976c5-59cb-4926-b9ac-019708a22d74 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:37.519422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73976c5-59cb-4926-b9ac-019708a22d74 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:37.519694 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73976c5-59cb-4926-b9ac-019708a22d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:37.536853 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-b73976c5-59cb-4926-b9ac-019708a22d74 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:37.538028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b73976c5-59cb-4926-b9ac-019708a22d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:37.602323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 351/705] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:37.602994 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73976c5-59cb-4926-b9ac-019708a22d74 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:37.617612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad791aff-74f9-4c33-853d-a7b4dc5bbb40 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:37.638376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:37.654506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5214c3e-7599-4a80-be3c-597379f6972b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:37.655109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5214c3e-7599-4a80-be3c-597379f6972b None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:37.655382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5214c3e-7599-4a80-be3c-597379f6972b None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:37.655657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5214c3e-7599-4a80-be3c-597379f6972b None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:37.656430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5214c3e-7599-4a80-be3c-597379f6972b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:37.656763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5214c3e-7599-4a80-be3c-597379f6972b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:37.658123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5214c3e-7599-4a80-be3c-597379f6972b None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:37.665725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:37.671226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 355/706] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:37.672098 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5214c3e-7599-4a80-be3c-597379f6972b None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:37.716760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73976c5-59cb-4926-b9ac-019708a22d74 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:37.763692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849049a-20b5-4b56-b3e6-458f8ff18fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:37.764322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849049a-20b5-4b56-b3e6-458f8ff18fff None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:37.764695 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849049a-20b5-4b56-b3e6-458f8ff18fff None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:37.764891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849049a-20b5-4b56-b3e6-458f8ff18fff None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:37.765485 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849049a-20b5-4b56-b3e6-458f8ff18fff None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:37.766838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849049a-20b5-4b56-b3e6-458f8ff18fff None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-f849049a-20b5-4b56-b3e6-458f8ff18fff None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:37.777975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:37.779018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:37.779869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:37.779869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:37.779869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:37.779869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:37.779869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:37.779869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:37.779869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:37.779869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:37.779869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:37.779869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:37.779869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:01:37.779869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:37.779869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 12:01:37.779869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:37.779869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 352/707] 10.222.0.27 () {56 vars in 1129 bytes} [Fri Aug 12 12:01:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:37.780675 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849049a-20b5-4b56-b3e6-458f8ff18fff None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:37.795149 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5214c3e-7599-4a80-be3c-597379f6972b None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:37.840183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21b6c3-d726-4d75-b944-a70f401520b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:37.841546 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21b6c3-d726-4d75-b944-a70f401520b1 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:37.842063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21b6c3-d726-4d75-b944-a70f401520b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:37.842539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21b6c3-d726-4d75-b944-a70f401520b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:37.850703 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21b6c3-d726-4d75-b944-a70f401520b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:37.852510 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21b6c3-d726-4d75-b944-a70f401520b1 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:37.886958 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:37.894644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849049a-20b5-4b56-b3e6-458f8ff18fff None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:37.930533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c31d0e-6481-42e8-acd7-add3ae1a9a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:37.931297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c31d0e-6481-42e8-acd7-add3ae1a9a23 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:37.931634 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c31d0e-6481-42e8-acd7-add3ae1a9a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:37.932293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c31d0e-6481-42e8-acd7-add3ae1a9a23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:37.939683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c31d0e-6481-42e8-acd7-add3ae1a9a23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:37.941802 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c31d0e-6481-42e8-acd7-add3ae1a9a23 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:37.983599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-de21b6c3-d726-4d75-b944-a70f401520b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:37.983599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:37.983599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:37.983599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:37.983599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:37.983599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:37.983599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:37.983599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:37.983599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:37.983599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:37.983599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:37.983599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:37.983599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:37.983599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:37.983599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:37.984784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:37.985691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:37.985691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:37.985691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:37.985691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:37.985691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:37.985691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:37.985691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:37.985691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:37.985691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 356/708] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:37.986227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21b6c3-d726-4d75-b944-a70f401520b1 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:38.053220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:38.069400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-b3c31d0e-6481-42e8-acd7-add3ae1a9a23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:38.069400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:38.069400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:38.069400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:38.069400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:38.069400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:38.069400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:38.069400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:38.069400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:38.069400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:38.069400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:38.069400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:38.069400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:38.069400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:38.069400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:38.070562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:38.071761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:38.071761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:38.071761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:38.071761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:38.071761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:38.071761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:38.071761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:38.071761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:38.071761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 353/709] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:38.071761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c31d0e-6481-42e8-acd7-add3ae1a9a23 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:38.099926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21b6c3-d726-4d75-b944-a70f401520b1 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:38.133170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a0793-91d1-42d8-80ec-7ec4dbf073cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:38.133699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a0793-91d1-42d8-80ec-7ec4dbf073cc None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:38.133981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a0793-91d1-42d8-80ec-7ec4dbf073cc None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:38.134250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a0793-91d1-42d8-80ec-7ec4dbf073cc None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:38.134879 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a0793-91d1-42d8-80ec-7ec4dbf073cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:38.136296 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a0793-91d1-42d8-80ec-7ec4dbf073cc None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:38.171412 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c31d0e-6481-42e8-acd7-add3ae1a9a23 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:38.203587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32564f-64e4-42f4-9b5b-78d64ffd099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:38.204164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32564f-64e4-42f4-9b5b-78d64ffd099b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:38.204441 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32564f-64e4-42f4-9b5b-78d64ffd099b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:38.204736 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32564f-64e4-42f4-9b5b-78d64ffd099b None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:38.205355 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32564f-64e4-42f4-9b5b-78d64ffd099b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:38.206260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32564f-64e4-42f4-9b5b-78d64ffd099b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:38.210191 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32564f-64e4-42f4-9b5b-78d64ffd099b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:38.211582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 357/710] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:01:37 2022] POST /identity/v3/users => generated 254 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:01:38.216806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 354/711] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Aug 12 12:01:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:38.217016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a0793-91d1-42d8-80ec-7ec4dbf073cc None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:38.217143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32564f-64e4-42f4-9b5b-78d64ffd099b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:38.240622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:38.333550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32564f-64e4-42f4-9b5b-78d64ffd099b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:38.341976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a0793-91d1-42d8-80ec-7ec4dbf073cc None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:38.358095 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:38.375382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da061a-0bd0-412e-9e2e-711c509ab9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:38.375983 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da061a-0bd0-412e-9e2e-711c509ab9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:38.376267 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da061a-0bd0-412e-9e2e-711c509ab9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:38.376520 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da061a-0bd0-412e-9e2e-711c509ab9b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:38.377236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da061a-0bd0-412e-9e2e-711c509ab9b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:38.378673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da061a-0bd0-412e-9e2e-711c509ab9b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:38.379722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da061a-0bd0-412e-9e2e-711c509ab9b5 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:38.383266 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ef632-51c6-46ff-9656-06c02721727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:38.383841 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ef632-51c6-46ff-9656-06c02721727d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:38.384122 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ef632-51c6-46ff-9656-06c02721727d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:38.384395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ef632-51c6-46ff-9656-06c02721727d None admin] PATH_INFO: `/v3/services/network` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:38.385024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ef632-51c6-46ff-9656-06c02721727d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:38.386813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ef632-51c6-46ff-9656-06c02721727d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:38.391890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 355/712] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:38.393034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da061a-0bd0-412e-9e2e-711c509ab9b5 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-9e7ef632-51c6-46ff-9656-06c02721727d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:38.403509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:38.404563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:38.405434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:38.405434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:38.405434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:38.405434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:38.405434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:38.405434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:38.405434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:38.405434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:38.405434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:38.405434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:38.405434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:01:38.405434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:38.405434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 12:01:38.405434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:38.405434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 358/713] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:38 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:38.406205 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ef632-51c6-46ff-9656-06c02721727d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:38.479454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:38.504319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da061a-0bd0-412e-9e2e-711c509ab9b5 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:38.514371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ef632-51c6-46ff-9656-06c02721727d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:38.538578 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4bfea-a87e-49be-9268-99d3d843c382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:38.539181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4bfea-a87e-49be-9268-99d3d843c382 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:38.539466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4bfea-a87e-49be-9268-99d3d843c382 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:38.539740 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4bfea-a87e-49be-9268-99d3d843c382 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:38.540429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4bfea-a87e-49be-9268-99d3d843c382 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:38.540767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4bfea-a87e-49be-9268-99d3d843c382 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:38.542184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4bfea-a87e-49be-9268-99d3d843c382 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:38.553438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea0b05-b122-43bc-a621-5ac153937797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:38.554592 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea0b05-b122-43bc-a621-5ac153937797 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:38.555062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 356/714] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:38.555171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea0b05-b122-43bc-a621-5ac153937797 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:38.555596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea0b05-b122-43bc-a621-5ac153937797 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:38.555872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4bfea-a87e-49be-9268-99d3d843c382 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:38.556603 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea0b05-b122-43bc-a621-5ac153937797 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:38.556932 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea0b05-b122-43bc-a621-5ac153937797 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:38.558311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea0b05-b122-43bc-a621-5ac153937797 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:38.564828 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 359/715] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Aug 12 12:01:38 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:38.566435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea0b05-b122-43bc-a621-5ac153937797 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:38.627213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:38.676212 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4bfea-a87e-49be-9268-99d3d843c382 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:38.699177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea0b05-b122-43bc-a621-5ac153937797 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:38.716410 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d56af-6cb4-4e9a-8618-aaac5b582c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:38.717072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d56af-6cb4-4e9a-8618-aaac5b582c1a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:38.717354 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d56af-6cb4-4e9a-8618-aaac5b582c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:38.717625 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d56af-6cb4-4e9a-8618-aaac5b582c1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:38.718366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d56af-6cb4-4e9a-8618-aaac5b582c1a None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=687d0375abd44d2481f988c6df90fad1) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:38.718672 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d56af-6cb4-4e9a-8618-aaac5b582c1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:38.720118 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d56af-6cb4-4e9a-8618-aaac5b582c1a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:38.728064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 357/716] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:38 2022] GET /identity/v3/role_assignments?user.id=687d0375abd44d2481f988c6df90fad1&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=906e3a8a6c7540a38f629f2567133347 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:38.729013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d56af-6cb4-4e9a-8618-aaac5b582c1a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:38.731781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:38.746536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a438cce-1145-4810-b087-9509b9ad88bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:38.747486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a438cce-1145-4810-b087-9509b9ad88bd None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:38.747978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a438cce-1145-4810-b087-9509b9ad88bd None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:38.748434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a438cce-1145-4810-b087-9509b9ad88bd None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:38.749307 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a438cce-1145-4810-b087-9509b9ad88bd None admin] RBAC: Adding query filter params (name=network) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:38.749789 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a438cce-1145-4810-b087-9509b9ad88bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:38.751447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a438cce-1145-4810-b087-9509b9ad88bd None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:38.757226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 360/717] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 12 12:01:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:38.763034 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a438cce-1145-4810-b087-9509b9ad88bd None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:38.887962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a438cce-1145-4810-b087-9509b9ad88bd None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:38.898129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:38.912030 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d56af-6cb4-4e9a-8618-aaac5b582c1a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:38.948105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b3f18-d18b-4095-8d08-484f2aa59fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:38.948754 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b3f18-d18b-4095-8d08-484f2aa59fbc None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:38.949099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b3f18-d18b-4095-8d08-484f2aa59fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:38.949389 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af87314-b2f3-40a3-adfc-69be28ceddb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:38.949521 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b3f18-d18b-4095-8d08-484f2aa59fbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:38.950005 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af87314-b2f3-40a3-adfc-69be28ceddb4 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:38.950287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af87314-b2f3-40a3-adfc-69be28ceddb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:38.950386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b3f18-d18b-4095-8d08-484f2aa59fbc None admin] RBAC: Adding query filter params (role.id=1416a05bf5ba4de88889c54ff5c58da2, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=5123401076a84ffcab49b00fee65d14d) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:38.950556 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af87314-b2f3-40a3-adfc-69be28ceddb4 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:38.950711 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b3f18-d18b-4095-8d08-484f2aa59fbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:38.951197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af87314-b2f3-40a3-adfc-69be28ceddb4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:38.951494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af87314-b2f3-40a3-adfc-69be28ceddb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:38.952347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b3f18-d18b-4095-8d08-484f2aa59fbc None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:38.952873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af87314-b2f3-40a3-adfc-69be28ceddb4 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:38.959215 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 358/718] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:38 2022] GET /identity/v3/role_assignments?user.id=5123401076a84ffcab49b00fee65d14d&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=1416a05bf5ba4de88889c54ff5c58da2 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:38.961970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 361/719] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 12 12:01:38 2022] GET /identity/v3/services?type=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:40.339402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4d9fd-74be-4115-8b01-9533169b0d5f None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:40.339922 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4d9fd-74be-4115-8b01-9533169b0d5f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:40.339922 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4d9fd-74be-4115-8b01-9533169b0d5f None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:40.341642 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 359/720] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:40.363078 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca8c45-515a-4605-b89d-2ebad92692ec None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:40.363580 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca8c45-515a-4605-b89d-2ebad92692ec None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:40.363685 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca8c45-515a-4605-b89d-2ebad92692ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:40.385572 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-74ca8c45-515a-4605-b89d-2ebad92692ec None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:40.386757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-74ca8c45-515a-4605-b89d-2ebad92692ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:40.472123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 362/721] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:40.792446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4d9fd-74be-4115-8b01-9533169b0d5f None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:40.886828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:40.933674 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4d9fd-74be-4115-8b01-9533169b0d5f None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:40.938142 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11004e78-5df8-4cf2-883b-23753ff4da12 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:40.938654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11004e78-5df8-4cf2-883b-23753ff4da12 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:40.939148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11004e78-5df8-4cf2-883b-23753ff4da12 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:40.941692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 363/722] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:40.946988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c6fc8-8681-4044-ba7b-36b3ff38bd2d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:40.947520 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c6fc8-8681-4044-ba7b-36b3ff38bd2d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:40.948004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c6fc8-8681-4044-ba7b-36b3ff38bd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:40.972745 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-b27c6fc8-8681-4044-ba7b-36b3ff38bd2d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:40.974398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b27c6fc8-8681-4044-ba7b-36b3ff38bd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:40.989790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8902b-59e2-495c-aa66-a092790527b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:40.993586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8902b-59e2-495c-aa66-a092790527b2 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:40.993586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8902b-59e2-495c-aa66-a092790527b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:40.993586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8902b-59e2-495c-aa66-a092790527b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:41.003835 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8902b-59e2-495c-aa66-a092790527b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-5bf8902b-59e2-495c-aa66-a092790527b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:41.010599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:41.015968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:41.016960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:41.016960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:41.016960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:41.016960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:41.016960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:41.016960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:41.016960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:41.016960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:41.016960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:41.016960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:41.016960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:41.016960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:41.016960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 360/723] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:41.016960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcc235-7384-4646-8a08-4556c5d927f2 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:41.017703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcc235-7384-4646-8a08-4556c5d927f2 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:41.017703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcc235-7384-4646-8a08-4556c5d927f2 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:41.017703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 361/724] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:41.017703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcc235-7384-4646-8a08-4556c5d927f2 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:41.081064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 364/725] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:41.082902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d009f-7a05-4666-a6fd-683150404fc6 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:41.083407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d009f-7a05-4666-a6fd-683150404fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:41.083927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d009f-7a05-4666-a6fd-683150404fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:41.115994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-2e8d009f-7a05-4666-a6fd-683150404fc6 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:41.117583 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2e8d009f-7a05-4666-a6fd-683150404fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:41.141119 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcc235-7384-4646-8a08-4556c5d927f2 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:41.161270 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:41.172404 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:41.182825 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9552d-a82d-4dba-a126-0af4b5a38bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:41.183425 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9552d-a82d-4dba-a126-0af4b5a38bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:41.183703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9552d-a82d-4dba-a126-0af4b5a38bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:41.183969 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9552d-a82d-4dba-a126-0af4b5a38bb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:41.184663 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9552d-a82d-4dba-a126-0af4b5a38bb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:41.184986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9552d-a82d-4dba-a126-0af4b5a38bb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:41.186405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9552d-a82d-4dba-a126-0af4b5a38bb4 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:41.196938 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 362/726] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:41.201907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9552d-a82d-4dba-a126-0af4b5a38bb4 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:41.264911 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 365/727] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:41.323229 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9552d-a82d-4dba-a126-0af4b5a38bb4 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:41.360060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836358ef-1a2d-4c15-a08f-985622f41ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:41.360689 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836358ef-1a2d-4c15-a08f-985622f41ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:41.360885 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836358ef-1a2d-4c15-a08f-985622f41ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:41.361156 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836358ef-1a2d-4c15-a08f-985622f41ac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:41.372536 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836358ef-1a2d-4c15-a08f-985622f41ac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:41.374009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836358ef-1a2d-4c15-a08f-985622f41ac2 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-836358ef-1a2d-4c15-a08f-985622f41ac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:41.393986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:41.398587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:41.399442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:41.399442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:41.399442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:41.399442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 363/728] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:01:41.401217 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d009f-7a05-4666-a6fd-683150404fc6 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:41.484994 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939dbf2-0009-4fba-9c36-cf673221c34c None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:41.485712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939dbf2-0009-4fba-9c36-cf673221c34c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:41.485991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939dbf2-0009-4fba-9c36-cf673221c34c None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:41.488123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 364/729] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:41.499818 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee8798-e0ff-4845-9af3-b5d49f1e337c None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:41.500098 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee8798-e0ff-4845-9af3-b5d49f1e337c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:41.500369 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee8798-e0ff-4845-9af3-b5d49f1e337c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:41.532465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-f1ee8798-e0ff-4845-9af3-b5d49f1e337c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:41.534263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f1ee8798-e0ff-4845-9af3-b5d49f1e337c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:41.574551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:41.595772 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d009f-7a05-4666-a6fd-683150404fc6 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:41.609383 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 365/730] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:41.610282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee8798-e0ff-4845-9af3-b5d49f1e337c None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:41.643860 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2cd8a-568d-4b27-9b56-a2a0507efaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:41.644876 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2cd8a-568d-4b27-9b56-a2a0507efaf6 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:41.645610 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2cd8a-568d-4b27-9b56-a2a0507efaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:41.646949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2cd8a-568d-4b27-9b56-a2a0507efaf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:41.647800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2cd8a-568d-4b27-9b56-a2a0507efaf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:41.648303 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2cd8a-568d-4b27-9b56-a2a0507efaf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:41.649920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2cd8a-568d-4b27-9b56-a2a0507efaf6 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:41.665912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 366/731] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:41.667067 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2cd8a-568d-4b27-9b56-a2a0507efaf6 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:41.693216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:41.733840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee8798-e0ff-4845-9af3-b5d49f1e337c None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:41.771613 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a998e50-fe5f-419f-8883-22d5c92737e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:41.772217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a998e50-fe5f-419f-8883-22d5c92737e3 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:41.772505 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a998e50-fe5f-419f-8883-22d5c92737e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:41.772815 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a998e50-fe5f-419f-8883-22d5c92737e3 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:41.773460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a998e50-fe5f-419f-8883-22d5c92737e3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:41.774811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a998e50-fe5f-419f-8883-22d5c92737e3 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:41.782288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2cd8a-568d-4b27-9b56-a2a0507efaf6 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:41.820404 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff505c-832f-450a-98a7-96a9ecb1ec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:41.821092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff505c-832f-450a-98a7-96a9ecb1ec64 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:41.821377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff505c-832f-450a-98a7-96a9ecb1ec64 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:41.821676 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff505c-832f-450a-98a7-96a9ecb1ec64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:41.831852 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff505c-832f-450a-98a7-96a9ecb1ec64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-d1ff505c-832f-450a-98a7-96a9ecb1ec64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:41.836972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:41.840187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:41.841053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:41.841053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:41.841053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:41.841053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:41.841053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:41.841053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:41.841053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:41.841053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:41.841053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:41.841053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:41.841053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:41.841053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:41.841053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 367/732] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:01:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:41.841053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff505c-832f-450a-98a7-96a9ecb1ec64 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:41.868954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 366/733] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 12 12:01:41 2022] POST /identity/v3/services => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:01:41.868954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d18aa6-ca7c-4edb-bb9e-31fbab3f420e None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:41.868954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d18aa6-ca7c-4edb-bb9e-31fbab3f420e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:41.868954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d18aa6-ca7c-4edb-bb9e-31fbab3f420e None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:41.868954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 367/734] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:41.868954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d18aa6-ca7c-4edb-bb9e-31fbab3f420e None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:41.908628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:41.939409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:41.962089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff505c-832f-450a-98a7-96a9ecb1ec64 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:42.001778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d18aa6-ca7c-4edb-bb9e-31fbab3f420e None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:42.007173 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8bb3b-e228-4d94-a404-9e11d5d94346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:42.007810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8bb3b-e228-4d94-a404-9e11d5d94346 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:42.008092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8bb3b-e228-4d94-a404-9e11d5d94346 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:42.008366 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8bb3b-e228-4d94-a404-9e11d5d94346 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:42.017081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8bb3b-e228-4d94-a404-9e11d5d94346 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:42.019067 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8bb3b-e228-4d94-a404-9e11d5d94346 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-40f8bb3b-e228-4d94-a404-9e11d5d94346 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:42.034129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:42.035634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:42.036533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:42.036533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:42.036533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:42.036533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:42.036533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:42.036533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:42.036533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:42.036533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:42.036533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 12:01:42.036533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:42.036533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 368/735] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 12 12:01:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:42.036533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb062a3-d953-48b4-978b-28a92e86f001 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:42.036533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb062a3-d953-48b4-978b-28a92e86f001 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:42.037309 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb062a3-d953-48b4-978b-28a92e86f001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:42.056888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-5bb062a3-d953-48b4-978b-28a92e86f001 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:42.058444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-5bb062a3-d953-48b4-978b-28a92e86f001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:42.060332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896ff13-e342-4ec3-83e7-9060851287ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:42.060988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896ff13-e342-4ec3-83e7-9060851287ac None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:42.061273 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896ff13-e342-4ec3-83e7-9060851287ac None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:42.061571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896ff13-e342-4ec3-83e7-9060851287ac None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:42.062232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896ff13-e342-4ec3-83e7-9060851287ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:42.062539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896ff13-e342-4ec3-83e7-9060851287ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:42.063912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896ff13-e342-4ec3-83e7-9060851287ac None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:42.073113 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 368/736] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:42.074150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896ff13-e342-4ec3-83e7-9060851287ac None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:42.135351 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 369/737] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:42 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 12 12:01:42.136558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb062a3-d953-48b4-978b-28a92e86f001 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:42.172099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:42.185302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896ff13-e342-4ec3-83e7-9060851287ac None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:42.220970 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f360c28-c330-4953-b5bf-19d7a5ac9400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:42.222038 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f360c28-c330-4953-b5bf-19d7a5ac9400 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:42.222495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f360c28-c330-4953-b5bf-19d7a5ac9400 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:42.222968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f360c28-c330-4953-b5bf-19d7a5ac9400 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:42.223776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f360c28-c330-4953-b5bf-19d7a5ac9400 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:42.225316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f360c28-c330-4953-b5bf-19d7a5ac9400 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:42.241255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:42.275885 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 369/738] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 12 12:01:42 2022] POST /identity/v3/services => generated 240 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:01:42.277243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f360c28-c330-4953-b5bf-19d7a5ac9400 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:42.292941 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb062a3-d953-48b4-978b-28a92e86f001 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:42.410207 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd6df0-0298-4f27-9beb-21f9e0c5dffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:42.410891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd6df0-0298-4f27-9beb-21f9e0c5dffd None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:42.411179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd6df0-0298-4f27-9beb-21f9e0c5dffd None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:42.411386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd6df0-0298-4f27-9beb-21f9e0c5dffd None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:42.412106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd6df0-0298-4f27-9beb-21f9e0c5dffd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:42.412409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd6df0-0298-4f27-9beb-21f9e0c5dffd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:42.413886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd6df0-0298-4f27-9beb-21f9e0c5dffd None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:42.417617 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f360c28-c330-4953-b5bf-19d7a5ac9400 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:42.441514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 370/739] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Aug 12 12:01:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:42.442537 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd6df0-0298-4f27-9beb-21f9e0c5dffd None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:42.464826 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0627c10-b133-423f-a575-2782efebd4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:42.465886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0627c10-b133-423f-a575-2782efebd4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:42.466389 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0627c10-b133-423f-a575-2782efebd4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:42.467567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0627c10-b133-423f-a575-2782efebd4c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:42.485236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0627c10-b133-423f-a575-2782efebd4c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:42.487125 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0627c10-b133-423f-a575-2782efebd4c7 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:42.498828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-e0627c10-b133-423f-a575-2782efebd4c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:42.508767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:42.510123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:42.511656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:42.511656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:42.511656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:42.511656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 370/740] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:42.511656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0627c10-b133-423f-a575-2782efebd4c7 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:42.579998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd6df0-0298-4f27-9beb-21f9e0c5dffd None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:42.609927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:42.629889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ce426-c2eb-438c-9ed5-5f6f21d5efac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:42.631063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ce426-c2eb-438c-9ed5-5f6f21d5efac None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:42.631828 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ce426-c2eb-438c-9ed5-5f6f21d5efac None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:42.632284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ce426-c2eb-438c-9ed5-5f6f21d5efac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:42.635500 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0627c10-b133-423f-a575-2782efebd4c7 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:42.645645 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ce426-c2eb-438c-9ed5-5f6f21d5efac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-f25ce426-c2eb-438c-9ed5-5f6f21d5efac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:42.652132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:42.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:42.654561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:42.654561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:42.654561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:42.654561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:42.654561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:42.654561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:42.654561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:42.654561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:42.654561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:42.654561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:42.654561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:42.654561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:42.654561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 371/741] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:01:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:42.655599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ce426-c2eb-438c-9ed5-5f6f21d5efac None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:42.686014 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b5cd2-d90d-40c6-9a6a-cbe5447a5004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:42.687650 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b5cd2-d90d-40c6-9a6a-cbe5447a5004 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:42.688112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b5cd2-d90d-40c6-9a6a-cbe5447a5004 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:42.688603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b5cd2-d90d-40c6-9a6a-cbe5447a5004 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:42.766865 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ce426-c2eb-438c-9ed5-5f6f21d5efac None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:42.781231 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:42.785265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b5cd2-d90d-40c6-9a6a-cbe5447a5004 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:42.787625 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b5cd2-d90d-40c6-9a6a-cbe5447a5004 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:42.814363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dba76a-3651-46d7-b923-73d1afd9cf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:42.815656 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dba76a-3651-46d7-b923-73d1afd9cf6d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:42.816152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dba76a-3651-46d7-b923-73d1afd9cf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:42.816610 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dba76a-3651-46d7-b923-73d1afd9cf6d None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:42.817464 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dba76a-3651-46d7-b923-73d1afd9cf6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:42.817963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dba76a-3651-46d7-b923-73d1afd9cf6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:42.819530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dba76a-3651-46d7-b923-73d1afd9cf6d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:42.832791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 372/742] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:42.833875 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dba76a-3651-46d7-b923-73d1afd9cf6d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:42.881637 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:42.911762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 371/743] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:01:42.911762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b5cd2-d90d-40c6-9a6a-cbe5447a5004 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:42.968373 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dba76a-3651-46d7-b923-73d1afd9cf6d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:43.010276 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa387b-f88a-475b-9557-ec1a90cafb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:43.011473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa387b-f88a-475b-9557-ec1a90cafb64 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:43.012433 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:43.012888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa387b-f88a-475b-9557-ec1a90cafb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:43.018757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa387b-f88a-475b-9557-ec1a90cafb64 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:43.018757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa387b-f88a-475b-9557-ec1a90cafb64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:43.018757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa387b-f88a-475b-9557-ec1a90cafb64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:43.019038 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b5cd2-d90d-40c6-9a6a-cbe5447a5004 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:43.019123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa387b-f88a-475b-9557-ec1a90cafb64 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:43.028753 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 373/744] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:43.029712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa387b-f88a-475b-9557-ec1a90cafb64 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:43.070628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154c42c-4716-4c74-a2e5-c38a1a62e4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:43.071413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154c42c-4716-4c74-a2e5-c38a1a62e4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:43.071793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154c42c-4716-4c74-a2e5-c38a1a62e4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:43.072165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154c42c-4716-4c74-a2e5-c38a1a62e4e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:43.081623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154c42c-4716-4c74-a2e5-c38a1a62e4e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:43.083105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154c42c-4716-4c74-a2e5-c38a1a62e4e6 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-e154c42c-4716-4c74-a2e5-c38a1a62e4e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:43.094575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:43.095497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:43.096583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:43.096583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:43.096583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:43.096583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:43.096583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:43.096583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:43.096583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:43.096583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:43.096583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:01:43.096583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:43.096583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 372/745] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 12 12:01:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:43.097198 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154c42c-4716-4c74-a2e5-c38a1a62e4e6 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:43.147633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa387b-f88a-475b-9557-ec1a90cafb64 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:43.149329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:43.183322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a261b-10bd-42a7-a4c1-8fa29ca0d3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:43.184330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a261b-10bd-42a7-a4c1-8fa29ca0d3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:43.184950 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a261b-10bd-42a7-a4c1-8fa29ca0d3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:43.185439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a261b-10bd-42a7-a4c1-8fa29ca0d3a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:43.198010 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a261b-10bd-42a7-a4c1-8fa29ca0d3a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:43.199914 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a261b-10bd-42a7-a4c1-8fa29ca0d3a8 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:43.202226 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154c42c-4716-4c74-a2e5-c38a1a62e4e6 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-997a261b-10bd-42a7-a4c1-8fa29ca0d3a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:43.215472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:43.216380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:43.217294 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:43.217294 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:43.217294 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:43.224447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 374/746] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:43.225640 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a261b-10bd-42a7-a4c1-8fa29ca0d3a8 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:43.237235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a7d5f-8335-47ac-83c9-8fed0241c8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:43.238061 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a7d5f-8335-47ac-83c9-8fed0241c8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:43.238172 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a7d5f-8335-47ac-83c9-8fed0241c8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:43.238383 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a7d5f-8335-47ac-83c9-8fed0241c8a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:43.249651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a7d5f-8335-47ac-83c9-8fed0241c8a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:43.251149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a7d5f-8335-47ac-83c9-8fed0241c8a4 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-207a7d5f-8335-47ac-83c9-8fed0241c8a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:43.266424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:43.268903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:43.268903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:43.268903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:43.273031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 373/747] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:43.273898 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a7d5f-8335-47ac-83c9-8fed0241c8a4 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:43.289825 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:43.340277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a261b-10bd-42a7-a4c1-8fa29ca0d3a8 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:43.378605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad8272-357b-4a51-a71b-0b823d0f66ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:43.379049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a7d5f-8335-47ac-83c9-8fed0241c8a4 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:43.379579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad8272-357b-4a51-a71b-0b823d0f66ab None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:43.379868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad8272-357b-4a51-a71b-0b823d0f66ab None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:43.380139 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad8272-357b-4a51-a71b-0b823d0f66ab None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:43.380907 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad8272-357b-4a51-a71b-0b823d0f66ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:43.381214 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad8272-357b-4a51-a71b-0b823d0f66ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:43.382652 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad8272-357b-4a51-a71b-0b823d0f66ab None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:43.401812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 375/748] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 12 12:01:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:43.402665 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad8272-357b-4a51-a71b-0b823d0f66ab None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:43.417881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:43.424668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9f1c7-954d-45a0-967d-f12911799561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:43.425286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9f1c7-954d-45a0-967d-f12911799561 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:43.425575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9f1c7-954d-45a0-967d-f12911799561 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:43.425843 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9f1c7-954d-45a0-967d-f12911799561 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:43.426493 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9f1c7-954d-45a0-967d-f12911799561 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:43.426798 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9f1c7-954d-45a0-967d-f12911799561 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:43.428172 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9f1c7-954d-45a0-967d-f12911799561 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:43.444676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 374/749] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:43.444676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9f1c7-954d-45a0-967d-f12911799561 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:43.516152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad8272-357b-4a51-a71b-0b823d0f66ab None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:43.529013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:43.554157 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebb45a-c82a-46fa-b4ef-866fd9789a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:43.562540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9f1c7-954d-45a0-967d-f12911799561 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:43.562662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebb45a-c82a-46fa-b4ef-866fd9789a2d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:43.562662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebb45a-c82a-46fa-b4ef-866fd9789a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:43.562662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebb45a-c82a-46fa-b4ef-866fd9789a2d None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:43.562662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebb45a-c82a-46fa-b4ef-866fd9789a2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:43.562662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebb45a-c82a-46fa-b4ef-866fd9789a2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:43.562662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebb45a-c82a-46fa-b4ef-866fd9789a2d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:43.574733 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 376/750] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:43.575833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebb45a-c82a-46fa-b4ef-866fd9789a2d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:43.599200 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4c388-b036-4022-8c94-e5f68f7faf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:43.599723 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4c388-b036-4022-8c94-e5f68f7faf82 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:43.600029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4c388-b036-4022-8c94-e5f68f7faf82 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:43.600308 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4c388-b036-4022-8c94-e5f68f7faf82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:43.610401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4c388-b036-4022-8c94-e5f68f7faf82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:43.611963 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4c388-b036-4022-8c94-e5f68f7faf82 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-55d4c388-b036-4022-8c94-e5f68f7faf82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:43.627165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:43.628124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:43.629049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:43.629049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:43.629049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:43.629049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 375/751] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:43.629712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4c388-b036-4022-8c94-e5f68f7faf82 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:43.666251 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:43.710844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebb45a-c82a-46fa-b4ef-866fd9789a2d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:43.740338 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4c388-b036-4022-8c94-e5f68f7faf82 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:43.755040 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448a1e4-3dae-4d6a-ae8d-23d13132c84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:43.755680 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448a1e4-3dae-4d6a-ae8d-23d13132c84f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:43.755961 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448a1e4-3dae-4d6a-ae8d-23d13132c84f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:43.756303 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448a1e4-3dae-4d6a-ae8d-23d13132c84f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:43.764990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448a1e4-3dae-4d6a-ae8d-23d13132c84f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:43.767182 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448a1e4-3dae-4d6a-ae8d-23d13132c84f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:43.773049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:43.778723 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e31796-1010-4067-be92-93852a069e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:43.779636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e31796-1010-4067-be92-93852a069e3d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:43.780116 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e31796-1010-4067-be92-93852a069e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:43.780593 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e31796-1010-4067-be92-93852a069e3d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:43.781021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-6448a1e4-3dae-4d6a-ae8d-23d13132c84f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:43.781021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:43.781021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:43.781021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:43.781021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:43.781021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:43.781021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:43.781021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:43.781021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:43.781021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:43.781021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:43.781021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:43.781021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:43.781021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:43.781021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:43.781868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:43.782807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:43.782807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:43.782807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:43.782807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:43.782807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:43.782807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:43.782807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:43.782807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:43.787129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 377/752] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:43.788579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448a1e4-3dae-4d6a-ae8d-23d13132c84f None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:43.794248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e31796-1010-4067-be92-93852a069e3d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:43.795906 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e31796-1010-4067-be92-93852a069e3d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-d5e31796-1010-4067-be92-93852a069e3d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:43.812241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:43.813279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:43.814180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:43.814180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:43.814180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:43.814180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:43.814180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:43.814180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:43.814180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:43.814180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:43.814180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 12 12:01:43.814180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:43.816999 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 376/753] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:01:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:43.818812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e31796-1010-4067-be92-93852a069e3d None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:43.900721 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448a1e4-3dae-4d6a-ae8d-23d13132c84f None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:43.934993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:43.940463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ef99e-a585-459e-93c1-2c349ea754e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:43.941147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ef99e-a585-459e-93c1-2c349ea754e9 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:43.941428 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ef99e-a585-459e-93c1-2c349ea754e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:43.941701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ef99e-a585-459e-93c1-2c349ea754e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:43.942481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ef99e-a585-459e-93c1-2c349ea754e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:43.942785 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ef99e-a585-459e-93c1-2c349ea754e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:43.944150 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ef99e-a585-459e-93c1-2c349ea754e9 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:43.947398 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e31796-1010-4067-be92-93852a069e3d None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:43.959951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 378/754] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:43.959951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ef99e-a585-459e-93c1-2c349ea754e9 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:43.987712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e149a-f12e-440a-a038-5d1fefda9ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:43.988438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e149a-f12e-440a-a038-5d1fefda9ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:43.988750 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e149a-f12e-440a-a038-5d1fefda9ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:43.989331 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e149a-f12e-440a-a038-5d1fefda9ff2 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:43.989896 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e149a-f12e-440a-a038-5d1fefda9ff2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:43.990119 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e149a-f12e-440a-a038-5d1fefda9ff2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:43.991550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e149a-f12e-440a-a038-5d1fefda9ff2 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:44.005631 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 377/755] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:44.007016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345d9bb-fa9f-4fdb-887e-f95055c5ce35 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:44.007307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345d9bb-fa9f-4fdb-887e-f95055c5ce35 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:44.007690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345d9bb-fa9f-4fdb-887e-f95055c5ce35 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:44.008909 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 378/756] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:44.013579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345d9bb-fa9f-4fdb-887e-f95055c5ce35 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:44.063234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:44.071112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ef99e-a585-459e-93c1-2c349ea754e9 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:44.106979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdb37d-20c1-4f61-89db-7ae426ee24dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:44.107584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdb37d-20c1-4f61-89db-7ae426ee24dc None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:44.107865 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdb37d-20c1-4f61-89db-7ae426ee24dc None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:44.108131 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdb37d-20c1-4f61-89db-7ae426ee24dc None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:44.109323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bdb37d-20c1-4f61-89db-7ae426ee24dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:44.109912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bdb37d-20c1-4f61-89db-7ae426ee24dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:44.112185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bdb37d-20c1-4f61-89db-7ae426ee24dc None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:44.122476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345d9bb-fa9f-4fdb-887e-f95055c5ce35 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:44.134842 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 379/757] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Aug 12 12:01:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:44.135729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdb37d-20c1-4f61-89db-7ae426ee24dc None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:44.161971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d90b7-6b43-44a4-8c41-798e1d99d5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:44.162674 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d90b7-6b43-44a4-8c41-798e1d99d5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:44.162961 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d90b7-6b43-44a4-8c41-798e1d99d5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:44.163284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d90b7-6b43-44a4-8c41-798e1d99d5c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:44.170631 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d90b7-6b43-44a4-8c41-798e1d99d5c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:44.172062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d90b7-6b43-44a4-8c41-798e1d99d5c0 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:44.183335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-656d90b7-6b43-44a4-8c41-798e1d99d5c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:44.183335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:44.183335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:44.183335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:44.183335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:44.183335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:44.183335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:44.183335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:44.183335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:44.183335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:44.183335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:44.183335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:44.183335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:44.183335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:44.183335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:44.184243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:44.185182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:44.185182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:44.185182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:44.185182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:44.185182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:44.185182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:44.185182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:44.185182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:44.185182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 379/758] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:44.186078 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd92ac4-b958-467e-b90f-e7b4585bc7be None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:44.186078 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd92ac4-b958-467e-b90f-e7b4585bc7be None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:44.186200 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd92ac4-b958-467e-b90f-e7b4585bc7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:44.194086 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:44.205876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-4fd92ac4-b958-467e-b90f-e7b4585bc7be None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:44.207078 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4fd92ac4-b958-467e-b90f-e7b4585bc7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:44.257696 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdb37d-20c1-4f61-89db-7ae426ee24dc None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:44.288998 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 380/759] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:01:44.288998 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd92ac4-b958-467e-b90f-e7b4585bc7be None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:44.296797 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227b101-8cd3-49d0-bb1b-4aa22b779a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:44.297377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227b101-8cd3-49d0-bb1b-4aa22b779a7e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:44.297660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227b101-8cd3-49d0-bb1b-4aa22b779a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:44.297932 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227b101-8cd3-49d0-bb1b-4aa22b779a7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:44.299117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227b101-8cd3-49d0-bb1b-4aa22b779a7e None admin] RBAC: Adding query filter params (role.id=1416a05bf5ba4de88889c54ff5c58da2, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=ab4e815bbe7f4ff880f4cc633da1ed8c) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:44.299230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227b101-8cd3-49d0-bb1b-4aa22b779a7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:44.300670 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227b101-8cd3-49d0-bb1b-4aa22b779a7e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:44.307119 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 380/760] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:44 2022] GET /identity/v3/role_assignments?user.id=ab4e815bbe7f4ff880f4cc633da1ed8c&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=1416a05bf5ba4de88889c54ff5c58da2 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:44.308068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227b101-8cd3-49d0-bb1b-4aa22b779a7e None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:44.342117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:44.417294 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd92ac4-b958-467e-b90f-e7b4585bc7be None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:44.426310 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227b101-8cd3-49d0-bb1b-4aa22b779a7e None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:44.453125 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:44.457917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702d910-749b-49e9-8ae7-0642cb372c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:44.458545 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702d910-749b-49e9-8ae7-0642cb372c7f None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:44.458818 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702d910-749b-49e9-8ae7-0642cb372c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:44.459089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702d910-749b-49e9-8ae7-0642cb372c7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:44.469331 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb1ac8-9cb9-42b5-ac74-f19364dbe891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:44.470317 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb1ac8-9cb9-42b5-ac74-f19364dbe891 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:44.470897 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb1ac8-9cb9-42b5-ac74-f19364dbe891 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:44.471456 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb1ac8-9cb9-42b5-ac74-f19364dbe891 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:44.472369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb1ac8-9cb9-42b5-ac74-f19364dbe891 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:44.473041 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb1ac8-9cb9-42b5-ac74-f19364dbe891 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:44.474752 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb1ac8-9cb9-42b5-ac74-f19364dbe891 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:44.490269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 381/761] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:44.491964 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f8f93-fccd-4148-bcc1-e01aa51b2cb2 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:44.492455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f8f93-fccd-4148-bcc1-e01aa51b2cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:44.492967 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f8f93-fccd-4148-bcc1-e01aa51b2cb2 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:44.494601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 382/762] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:44.496056 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f8f93-fccd-4148-bcc1-e01aa51b2cb2 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:44.533740 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702d910-749b-49e9-8ae7-0642cb372c7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:44.535252 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702d910-749b-49e9-8ae7-0642cb372c7f None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:44.574994 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:44.620040 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f8f93-fccd-4148-bcc1-e01aa51b2cb2 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:44.650300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 381/763] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:01:44.651561 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26377ded-0d13-4258-9ab3-b7705f616382 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:44.651853 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26377ded-0d13-4258-9ab3-b7705f616382 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:44.652133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26377ded-0d13-4258-9ab3-b7705f616382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:44.656852 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc82749-8fcc-4c53-9a0f-d64ad8fda855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:44.657618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc82749-8fcc-4c53-9a0f-d64ad8fda855 None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:44.658774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc82749-8fcc-4c53-9a0f-d64ad8fda855 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:44.659157 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc82749-8fcc-4c53-9a0f-d64ad8fda855 None admin] PATH_INFO: `/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/687d0375abd44d2481f988c6df90fad1/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:44.662043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc82749-8fcc-4c53-9a0f-d64ad8fda855 None admin] RBAC: Authorizing `identity:create_grant(project_id=20c5570631a34c4ab3868eb1cd96e345, user_id=687d0375abd44d2481f988c6df90fad1, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:44.663574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc82749-8fcc-4c53-9a0f-d64ad8fda855 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:44.669937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-26377ded-0d13-4258-9ab3-b7705f616382 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:44.671128 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-26377ded-0d13-4258-9ab3-b7705f616382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:44.737708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 382/764] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:01:44.738894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26377ded-0d13-4258-9ab3-b7705f616382 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:44.748184 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 383/765] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:01:44 2022] PUT /identity/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/687d0375abd44d2481f988c6df90fad1/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:01:44.749278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc82749-8fcc-4c53-9a0f-d64ad8fda855 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:44.817376 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:44.877747 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:44.939528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26377ded-0d13-4258-9ab3-b7705f616382 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:44.944833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc82749-8fcc-4c53-9a0f-d64ad8fda855 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:44.988018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92fee4-5d83-4b49-9ab4-95bf0c3189ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:44.988819 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92fee4-5d83-4b49-9ab4-95bf0c3189ec None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:44.989099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92fee4-5d83-4b49-9ab4-95bf0c3189ec None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:44.989367 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92fee4-5d83-4b49-9ab4-95bf0c3189ec None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:44.990059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92fee4-5d83-4b49-9ab4-95bf0c3189ec None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:44.991816 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92fee4-5d83-4b49-9ab4-95bf0c3189ec None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:44.992551 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1001db-587b-41fb-ac5b-7b47886ada17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:44.993651 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1001db-587b-41fb-ac5b-7b47886ada17 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:44.994140 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1001db-587b-41fb-ac5b-7b47886ada17 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:44.994846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1001db-587b-41fb-ac5b-7b47886ada17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-0c92fee4-5d83-4b49-9ab4-95bf0c3189ec None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:45.001382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:45.002376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:45.003394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:45.003394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:45.003394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:45.003394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:45.003394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:45.003394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:45.003394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:45.003394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:45.003394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:45.003394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:45.003394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:01:45.003394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:45.003394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 12:01:45.003394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:45.008052 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 383/766] 10.222.0.27 () {56 vars in 1129 bytes} [Fri Aug 12 12:01:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:45.009391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92fee4-5d83-4b49-9ab4-95bf0c3189ec None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:45.012246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1001db-587b-41fb-ac5b-7b47886ada17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:45.016657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1001db-587b-41fb-ac5b-7b47886ada17 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-2e1001db-587b-41fb-ac5b-7b47886ada17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:45.037396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:45.038370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:45.039323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:45.039323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:45.039323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:45.041674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 384/767] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:45.042636 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1001db-587b-41fb-ac5b-7b47886ada17 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:45.120836 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92fee4-5d83-4b49-9ab4-95bf0c3189ec None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:45.148835 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:45.154295 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:45.162755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217199f-36c7-4cc0-b457-19e4df3c1c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:45.163576 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217199f-36c7-4cc0-b457-19e4df3c1c01 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:45.163858 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217199f-36c7-4cc0-b457-19e4df3c1c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:45.164438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217199f-36c7-4cc0-b457-19e4df3c1c01 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:45.171793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217199f-36c7-4cc0-b457-19e4df3c1c01 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:45.171793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217199f-36c7-4cc0-b457-19e4df3c1c01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:45.171793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217199f-36c7-4cc0-b457-19e4df3c1c01 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:45.172053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1001db-587b-41fb-ac5b-7b47886ada17 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:45.179440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 384/768] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Aug 12 12:01:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:45.180424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217199f-36c7-4cc0-b457-19e4df3c1c01 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:45.208375 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51038ae-c475-4e65-b870-265658f17c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:45.209137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51038ae-c475-4e65-b870-265658f17c79 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:45.209481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51038ae-c475-4e65-b870-265658f17c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:45.209820 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51038ae-c475-4e65-b870-265658f17c79 None admin] PATH_INFO: `/v3/services/network` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:45.211171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51038ae-c475-4e65-b870-265658f17c79 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:45.212014 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51038ae-c475-4e65-b870-265658f17c79 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-a51038ae-c475-4e65-b870-265658f17c79 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:45.221893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:45.222984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:45.223791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:45.223791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:45.223791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:45.223791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:45.223791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:45.223791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:45.223791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:45.223791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:45.223791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:45.223791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:45.223791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:01:45.223791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:45.223791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 12:01:45.223791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:45.224512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 385/769] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:45 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:45.229411 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51038ae-c475-4e65-b870-265658f17c79 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:45.294405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217199f-36c7-4cc0-b457-19e4df3c1c01 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:45.331728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2049f-0272-4d92-a7d2-351ac35c1423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:45.332360 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2049f-0272-4d92-a7d2-351ac35c1423 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:45.332662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2049f-0272-4d92-a7d2-351ac35c1423 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:45.332934 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2049f-0272-4d92-a7d2-351ac35c1423 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:45.333609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2049f-0272-4d92-a7d2-351ac35c1423 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:45.333983 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2049f-0272-4d92-a7d2-351ac35c1423 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:45.335451 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2049f-0272-4d92-a7d2-351ac35c1423 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:45.337422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51038ae-c475-4e65-b870-265658f17c79 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:45.352461 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 385/770] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:45.353586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2049f-0272-4d92-a7d2-351ac35c1423 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:45.374027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd4706-8b0d-42b1-9192-696b303fe38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:45.374643 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd4706-8b0d-42b1-9192-696b303fe38e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:45.374929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd4706-8b0d-42b1-9192-696b303fe38e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:45.375208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd4706-8b0d-42b1-9192-696b303fe38e None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:45.375906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd4706-8b0d-42b1-9192-696b303fe38e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:45.376216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd4706-8b0d-42b1-9192-696b303fe38e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:45.377780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd4706-8b0d-42b1-9192-696b303fe38e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:45.383404 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 386/771] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Aug 12 12:01:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:45.384285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd4706-8b0d-42b1-9192-696b303fe38e None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:45.404348 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:45.446855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:45.472537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2049f-0272-4d92-a7d2-351ac35c1423 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:45.498371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd4706-8b0d-42b1-9192-696b303fe38e None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:45.520223 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9e439-2e38-41db-81a1-79fe8fae44f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:45.520924 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9e439-2e38-41db-81a1-79fe8fae44f9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:45.521235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9e439-2e38-41db-81a1-79fe8fae44f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:45.521507 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9e439-2e38-41db-81a1-79fe8fae44f9 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:45.522154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9e439-2e38-41db-81a1-79fe8fae44f9 None admin] RBAC: Adding query filter params (name=network) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:45.522967 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9e439-2e38-41db-81a1-79fe8fae44f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:45.524615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9e439-2e38-41db-81a1-79fe8fae44f9 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:45.531387 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 386/772] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 12 12:01:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:45.532199 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9e439-2e38-41db-81a1-79fe8fae44f9 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:45.542983 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50205fbf-6a4e-4594-9d5b-e4cf54efe3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:45.543726 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50205fbf-6a4e-4594-9d5b-e4cf54efe3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:45.544147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50205fbf-6a4e-4594-9d5b-e4cf54efe3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:45.544679 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50205fbf-6a4e-4594-9d5b-e4cf54efe3c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:45.552500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50205fbf-6a4e-4594-9d5b-e4cf54efe3c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:45.554596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50205fbf-6a4e-4594-9d5b-e4cf54efe3c0 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:45.568430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-50205fbf-6a4e-4594-9d5b-e4cf54efe3c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:45.568430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:45.568430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:45.568430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:45.568430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:45.568430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:45.568430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:45.568430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:45.568430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:45.568430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:45.568430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:45.568430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:45.568430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:45.568430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:45.568430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:45.569509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:45.571287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:45.571287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:45.571287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:45.571287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:45.571287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:45.571287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:45.571287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:45.571287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:45.576580 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 387/773] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:45.577691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50205fbf-6a4e-4594-9d5b-e4cf54efe3c0 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:45.646057 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9e439-2e38-41db-81a1-79fe8fae44f9 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:45.664318 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:45.668057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:45.685941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf467c7-35f4-44f7-9741-f93d03a8567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:45.686558 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf467c7-35f4-44f7-9741-f93d03a8567b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:45.686916 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf467c7-35f4-44f7-9741-f93d03a8567b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:45.687259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf467c7-35f4-44f7-9741-f93d03a8567b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:45.688032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf467c7-35f4-44f7-9741-f93d03a8567b None admin] RBAC: Adding query filter params (interface=public, service_id=df96479d3379428bb81c8651762b116b, region_id=RegionOne) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:45.688454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf467c7-35f4-44f7-9741-f93d03a8567b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:45.690648 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf467c7-35f4-44f7-9741-f93d03a8567b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:45.702687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 387/774] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Aug 12 12:01:45 2022] GET /identity/v3/endpoints?service_id=df96479d3379428bb81c8651762b116b&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:45.702687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf467c7-35f4-44f7-9741-f93d03a8567b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:45.702913 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50205fbf-6a4e-4594-9d5b-e4cf54efe3c0 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:45.744716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6b9e2-7fdd-435e-ab99-d8bcff702270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:45.745286 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6b9e2-7fdd-435e-ab99-d8bcff702270 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:45.745572 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6b9e2-7fdd-435e-ab99-d8bcff702270 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:45.745848 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6b9e2-7fdd-435e-ab99-d8bcff702270 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:45.746487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6b9e2-7fdd-435e-ab99-d8bcff702270 None admin] RBAC: Adding query filter params (type=network) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:45.746794 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6b9e2-7fdd-435e-ab99-d8bcff702270 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:45.748331 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6b9e2-7fdd-435e-ab99-d8bcff702270 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:45.759102 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 388/775] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 12 12:01:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:45.760242 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6b9e2-7fdd-435e-ab99-d8bcff702270 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:45.837377 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf467c7-35f4-44f7-9741-f93d03a8567b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:45.883014 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f193f-5dfa-488a-a2e8-5460ce7dfb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:45.884157 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f193f-5dfa-488a-a2e8-5460ce7dfb77 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:45.884617 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f193f-5dfa-488a-a2e8-5460ce7dfb77 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:45.885087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f193f-5dfa-488a-a2e8-5460ce7dfb77 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:45.885732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6b9e2-7fdd-435e-ab99-d8bcff702270 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:45.886172 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f193f-5dfa-488a-a2e8-5460ce7dfb77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:45.886662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f193f-5dfa-488a-a2e8-5460ce7dfb77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:45.889239 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f193f-5dfa-488a-a2e8-5460ce7dfb77 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:45.907689 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 388/776] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:45.908559 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f193f-5dfa-488a-a2e8-5460ce7dfb77 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:45.930573 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef2dbb-cdf9-4f44-990e-4099efde1d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:45.931514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef2dbb-cdf9-4f44-990e-4099efde1d47 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:45.932010 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef2dbb-cdf9-4f44-990e-4099efde1d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:45.932938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef2dbb-cdf9-4f44-990e-4099efde1d47 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:45.934024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef2dbb-cdf9-4f44-990e-4099efde1d47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:45.935763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef2dbb-cdf9-4f44-990e-4099efde1d47 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:45.952923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 389/777] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 12 12:01:45 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:45.952923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef2dbb-cdf9-4f44-990e-4099efde1d47 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:45.959618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:45.977952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:46.094710 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f193f-5dfa-488a-a2e8-5460ce7dfb77 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:46.150318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef2dbb-cdf9-4f44-990e-4099efde1d47 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:46.159348 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac895885-953f-46bd-a868-bb37ea13ca96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:46.159962 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac895885-953f-46bd-a868-bb37ea13ca96 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:46.160245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac895885-953f-46bd-a868-bb37ea13ca96 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:46.160515 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac895885-953f-46bd-a868-bb37ea13ca96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:46.161276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac895885-953f-46bd-a868-bb37ea13ca96 None admin] RBAC: Adding query filter params (interface=public, service_id=67a5b2db44084723b24649bd000cb73e, region_id=RegionOne) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:46.161586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac895885-953f-46bd-a868-bb37ea13ca96 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:46.163086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac895885-953f-46bd-a868-bb37ea13ca96 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:46.170270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 389/778] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Aug 12 12:01:45 2022] GET /identity/v3/endpoints?service_id=67a5b2db44084723b24649bd000cb73e&interface=public®ion_id=RegionOne => generated 194 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:46.176221 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac895885-953f-46bd-a868-bb37ea13ca96 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:46.194798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead071c0-a250-4041-9a19-f28e5318698d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:46.195720 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead071c0-a250-4041-9a19-f28e5318698d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:46.196196 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead071c0-a250-4041-9a19-f28e5318698d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:46.196684 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead071c0-a250-4041-9a19-f28e5318698d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:46.197634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead071c0-a250-4041-9a19-f28e5318698d None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=5123401076a84ffcab49b00fee65d14d) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:46.198138 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead071c0-a250-4041-9a19-f28e5318698d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:46.199779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead071c0-a250-4041-9a19-f28e5318698d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:46.206422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 390/779] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:45 2022] GET /identity/v3/role_assignments?user.id=5123401076a84ffcab49b00fee65d14d&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=906e3a8a6c7540a38f629f2567133347 => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:46.326364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac895885-953f-46bd-a868-bb37ea13ca96 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:46.333938 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:46.351626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1228d5-8556-4604-95b2-8a2be4b2de86 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:46.357883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1228d5-8556-4604-95b2-8a2be4b2de86 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:46.357883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1228d5-8556-4604-95b2-8a2be4b2de86 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:46.357883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 391/780] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:01:46.359681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a3b9a-e123-41ff-8c14-55286132b67c None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:46.360171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a3b9a-e123-41ff-8c14-55286132b67c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:46.360659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a3b9a-e123-41ff-8c14-55286132b67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:46.381221 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-669a3b9a-e123-41ff-8c14-55286132b67c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:46.382762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-669a3b9a-e123-41ff-8c14-55286132b67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:46.387294 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8f7da-a0be-4a26-a6a8-8bcb3e8ba2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:46.387908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8f7da-a0be-4a26-a6a8-8bcb3e8ba2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:46.388188 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8f7da-a0be-4a26-a6a8-8bcb3e8ba2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:46.388459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8f7da-a0be-4a26-a6a8-8bcb3e8ba2b8 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:46.389097 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8f7da-a0be-4a26-a6a8-8bcb3e8ba2b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:46.390871 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8f7da-a0be-4a26-a6a8-8bcb3e8ba2b8 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:46.400590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 390/781] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 12 12:01:46 2022] GET /identity/v3/services => generated 772 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:46.465612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:46.479565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 392/782] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:01:46.809738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8f7da-a0be-4a26-a6a8-8bcb3e8ba2b8 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:46.815101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1713b-e873-4716-a177-7839cf2ff424 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:46.815101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1713b-e873-4716-a177-7839cf2ff424 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:46.815101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1713b-e873-4716-a177-7839cf2ff424 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:46.816629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 393/783] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:46.821780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573430d-88b1-4cea-83c8-0c6208076d0b None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:46.822060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573430d-88b1-4cea-83c8-0c6208076d0b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:46.822333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573430d-88b1-4cea-83c8-0c6208076d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:46.843320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f573430d-88b1-4cea-83c8-0c6208076d0b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:46.844939 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f573430d-88b1-4cea-83c8-0c6208076d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:46.930889 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8f7da-a0be-4a26-a6a8-8bcb3e8ba2b8 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:46.966899 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bda2bd-146f-4feb-b0a5-12d8b036a17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:46.967893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bda2bd-146f-4feb-b0a5-12d8b036a17c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:46.968399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bda2bd-146f-4feb-b0a5-12d8b036a17c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:46.968907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bda2bd-146f-4feb-b0a5-12d8b036a17c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:46.978909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 394/784] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:01:46.980918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bda2bd-146f-4feb-b0a5-12d8b036a17c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-45bda2bd-146f-4feb-b0a5-12d8b036a17c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:46.986006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:46.987317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:46.988142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:46.988142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:46.988142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:46.988142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:46.988142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:46.988142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:46.988142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:46.988142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:46.988142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:46.988142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:46.988142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:46.988142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:46.990755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 391/785] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:46.996770 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573430d-88b1-4cea-83c8-0c6208076d0b None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:47.139768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573430d-88b1-4cea-83c8-0c6208076d0b None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:47.179263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:47.188338 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f9f5b-3b8a-44de-9bb4-b34e268952a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:47.189268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f9f5b-3b8a-44de-9bb4-b34e268952a1 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:47.195925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f9f5b-3b8a-44de-9bb4-b34e268952a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:47.195925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f9f5b-3b8a-44de-9bb4-b34e268952a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:47.195925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f9f5b-3b8a-44de-9bb4-b34e268952a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:47.195925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f9f5b-3b8a-44de-9bb4-b34e268952a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:47.195925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f9f5b-3b8a-44de-9bb4-b34e268952a1 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:47.207736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 395/786] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:47.213299 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bda2bd-146f-4feb-b0a5-12d8b036a17c None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:47.245099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:47.320229 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bda2bd-146f-4feb-b0a5-12d8b036a17c None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:47.356157 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6d146-6d19-412e-91c4-d3a08fa7f9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:47.357203 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6d146-6d19-412e-91c4-d3a08fa7f9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:47.357692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6d146-6d19-412e-91c4-d3a08fa7f9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:47.358206 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6d146-6d19-412e-91c4-d3a08fa7f9f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:47.369499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6d146-6d19-412e-91c4-d3a08fa7f9f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:47.371193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6d146-6d19-412e-91c4-d3a08fa7f9f1 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-a8c6d146-6d19-412e-91c4-d3a08fa7f9f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:47.386779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:47.387758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:47.388708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:47.388708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:47.388708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:47.390839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 392/787] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:47.397743 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f9f5b-3b8a-44de-9bb4-b34e268952a1 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:47.437367 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6d146-6d19-412e-91c4-d3a08fa7f9f1 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:47.588371 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:47.597148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f9f5b-3b8a-44de-9bb4-b34e268952a1 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:47.628046 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6d146-6d19-412e-91c4-d3a08fa7f9f1 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:47.663331 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f3055-87fa-4a1e-9258-67ec552bcb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:47.664229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f3055-87fa-4a1e-9258-67ec552bcb10 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:47.665717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f3055-87fa-4a1e-9258-67ec552bcb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:47.667532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f3055-87fa-4a1e-9258-67ec552bcb10 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:47.668391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f3055-87fa-4a1e-9258-67ec552bcb10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:47.668905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f3055-87fa-4a1e-9258-67ec552bcb10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:47.672416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f3055-87fa-4a1e-9258-67ec552bcb10 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:47.695717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f526dd-a84e-4eee-8cde-906128cc14b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:47.696677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f526dd-a84e-4eee-8cde-906128cc14b6 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:47.697184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f526dd-a84e-4eee-8cde-906128cc14b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:47.697639 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f526dd-a84e-4eee-8cde-906128cc14b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:47.702855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 396/788] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:47.708846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f3055-87fa-4a1e-9258-67ec552bcb10 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:47.709743 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f526dd-a84e-4eee-8cde-906128cc14b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-36f526dd-a84e-4eee-8cde-906128cc14b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:47.715245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:47.716214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:47.717079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:47.717079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:47.717079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:47.717079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:47.717079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:47.717079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:47.717079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:47.717079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:47.717079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:47.717079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:47.717079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:47.717079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:47.719281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 393/789] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:01:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:01:47.725705 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f526dd-a84e-4eee-8cde-906128cc14b6 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:47.788465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:47.880131 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f3055-87fa-4a1e-9258-67ec552bcb10 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:47.911227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f526dd-a84e-4eee-8cde-906128cc14b6 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:47.944532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860b396-c8a3-4422-a66a-273e989a706f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:47.945090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860b396-c8a3-4422-a66a-273e989a706f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:47.945366 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860b396-c8a3-4422-a66a-273e989a706f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:47.945635 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860b396-c8a3-4422-a66a-273e989a706f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:47.955742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860b396-c8a3-4422-a66a-273e989a706f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:47.957217 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860b396-c8a3-4422-a66a-273e989a706f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-7860b396-c8a3-4422-a66a-273e989a706f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:47.975492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:47.976681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:47.977626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:47.977626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:47.977626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:47.977626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:47.977626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:47.977626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:47.977626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:47.977626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:47.977626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 12:01:47.977626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:47.977626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 397/790] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 12 12:01:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:01:47.980141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4ca2c-4503-41bf-b736-b2cecb062075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:47.980845 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4ca2c-4503-41bf-b736-b2cecb062075 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:47.981191 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4ca2c-4503-41bf-b736-b2cecb062075 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:47.981530 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4ca2c-4503-41bf-b736-b2cecb062075 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:47.982497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4ca2c-4503-41bf-b736-b2cecb062075 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:47.982804 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4ca2c-4503-41bf-b736-b2cecb062075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:47.984165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4ca2c-4503-41bf-b736-b2cecb062075 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:47.993114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860b396-c8a3-4422-a66a-273e989a706f None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:47.996970 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 394/791] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:48.003754 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4ca2c-4503-41bf-b736-b2cecb062075 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:48.006841 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:48.181380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860b396-c8a3-4422-a66a-273e989a706f None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:48.192150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4ca2c-4503-41bf-b736-b2cecb062075 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:48.196154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:48.243142 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31545947-974c-4a85-81a9-472cfb61aabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:48.243748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31545947-974c-4a85-81a9-472cfb61aabd None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:48.244026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31545947-974c-4a85-81a9-472cfb61aabd None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:48.244295 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31545947-974c-4a85-81a9-472cfb61aabd None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:48.245058 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31545947-974c-4a85-81a9-472cfb61aabd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:48.245359 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31545947-974c-4a85-81a9-472cfb61aabd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:48.246791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31545947-974c-4a85-81a9-472cfb61aabd None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:48.255535 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68177dfb-e2bc-48e2-9bb8-2f74d4e8f429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:48.256232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68177dfb-e2bc-48e2-9bb8-2f74d4e8f429 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:48.256511 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68177dfb-e2bc-48e2-9bb8-2f74d4e8f429 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:48.256802 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68177dfb-e2bc-48e2-9bb8-2f74d4e8f429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:48.268883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68177dfb-e2bc-48e2-9bb8-2f74d4e8f429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:48.270404 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68177dfb-e2bc-48e2-9bb8-2f74d4e8f429 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:48.277581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 398/792] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Aug 12 12:01:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:48.284814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31545947-974c-4a85-81a9-472cfb61aabd None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-68177dfb-e2bc-48e2-9bb8-2f74d4e8f429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:48.287737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:48.288683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:48.289571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:48.289571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:48.289571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:48.289571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 395/793] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:01:48.295983 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68177dfb-e2bc-48e2-9bb8-2f74d4e8f429 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:48.393847 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:48.458361 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68177dfb-e2bc-48e2-9bb8-2f74d4e8f429 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:48.482498 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31545947-974c-4a85-81a9-472cfb61aabd None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:48.497256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a84569-a40e-41f3-be72-f08511f8eaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:48.498049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a84569-a40e-41f3-be72-f08511f8eaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:48.498336 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a84569-a40e-41f3-be72-f08511f8eaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:48.498609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a84569-a40e-41f3-be72-f08511f8eaf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:48.499475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a84569-a40e-41f3-be72-f08511f8eaf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:48.500162 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a84569-a40e-41f3-be72-f08511f8eaf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:48.502418 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a84569-a40e-41f3-be72-f08511f8eaf1 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:48.518729 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 396/794] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:48.520091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3bfab-3257-4eb3-a264-c340df0bb1bb None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:48.520380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3bfab-3257-4eb3-a264-c340df0bb1bb None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:48.520704 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3bfab-3257-4eb3-a264-c340df0bb1bb None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:48.522210 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 397/795] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:48.523308 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fa4ab-698d-4e53-96ad-097525bc4e68 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:48.523653 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fa4ab-698d-4e53-96ad-097525bc4e68 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:48.523929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fa4ab-698d-4e53-96ad-097525bc4e68 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:48.524290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d158df1-9d34-480d-b5b9-78c871a80d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:48.525180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d158df1-9d34-480d-b5b9-78c871a80d8f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:48.525371 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 398/796] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:48.525833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d158df1-9d34-480d-b5b9-78c871a80d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:48.526073 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fa4ab-698d-4e53-96ad-097525bc4e68 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:48.530387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d158df1-9d34-480d-b5b9-78c871a80d8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:48.540252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d158df1-9d34-480d-b5b9-78c871a80d8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:48.541742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d158df1-9d34-480d-b5b9-78c871a80d8f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-5d158df1-9d34-480d-b5b9-78c871a80d8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:48.558766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:48.559710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:48.560575 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:48.560575 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:48.560575 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:48.560575 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 399/797] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:48.561444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b4f2a-3990-4e9a-81d1-0635fd74ee6d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:48.561444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b4f2a-3990-4e9a-81d1-0635fd74ee6d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:48.561640 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b4f2a-3990-4e9a-81d1-0635fd74ee6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:48.582032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-cd6b4f2a-3990-4e9a-81d1-0635fd74ee6d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:48.583213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-cd6b4f2a-3990-4e9a-81d1-0635fd74ee6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:48.597850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:48.647106 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fa4ab-698d-4e53-96ad-097525bc4e68 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:48.664663 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 400/798] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:01:48.664663 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bd324-9372-4875-ab6f-a7d592c48436 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:48.664663 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bd324-9372-4875-ab6f-a7d592c48436 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:48.664663 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bd324-9372-4875-ab6f-a7d592c48436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:48.675811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:48.686422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-383bd324-9372-4875-ab6f-a7d592c48436 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:48.687994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-383bd324-9372-4875-ab6f-a7d592c48436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:48.697820 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3786f-eb21-4291-9347-ae66772ce6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:48.698546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3786f-eb21-4291-9347-ae66772ce6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:48.698892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3786f-eb21-4291-9347-ae66772ce6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:48.699265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3786f-eb21-4291-9347-ae66772ce6e8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:48.707563 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3786f-eb21-4291-9347-ae66772ce6e8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:48.709733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3786f-eb21-4291-9347-ae66772ce6e8 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-26b3786f-eb21-4291-9347-ae66772ce6e8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:48.722748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:48.723894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:48.725090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:48.725090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:48.725090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:48.725090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:48.725090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:48.725090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:48.725090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:48.725090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:48.725090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:01:48.725090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:48.726877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 399/799] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 12 12:01:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:48.727788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3786f-eb21-4291-9347-ae66772ce6e8 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:48.776685 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 401/800] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:01:48.777037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bd324-9372-4875-ab6f-a7d592c48436 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:48.851708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3786f-eb21-4291-9347-ae66772ce6e8 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:48.868982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:48.885701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bd324-9372-4875-ab6f-a7d592c48436 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:48.897090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b34f2-04e5-49b6-9810-3139500b8db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:48.898054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b34f2-04e5-49b6-9810-3139500b8db8 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:48.901864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b34f2-04e5-49b6-9810-3139500b8db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:48.901864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b34f2-04e5-49b6-9810-3139500b8db8 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:48.901864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b34f2-04e5-49b6-9810-3139500b8db8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:48.901864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b34f2-04e5-49b6-9810-3139500b8db8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:48.902128 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b34f2-04e5-49b6-9810-3139500b8db8 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:48.918543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 400/801] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:48.919774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69bbe2-c503-4ae0-a1eb-9e788c31c287 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:48.920058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69bbe2-c503-4ae0-a1eb-9e788c31c287 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:48.920382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69bbe2-c503-4ae0-a1eb-9e788c31c287 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:48.926143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 401/802] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:48.927099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69bbe2-c503-4ae0-a1eb-9e788c31c287 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:48.929372 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60476f8-99bb-4570-a3b5-064c50b042ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:48.930170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60476f8-99bb-4570-a3b5-064c50b042ae None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:48.930522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60476f8-99bb-4570-a3b5-064c50b042ae None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:48.930961 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60476f8-99bb-4570-a3b5-064c50b042ae None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:48.931770 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60476f8-99bb-4570-a3b5-064c50b042ae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:48.932159 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60476f8-99bb-4570-a3b5-064c50b042ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:48.933811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60476f8-99bb-4570-a3b5-064c50b042ae None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:48.952110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 402/803] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 12 12:01:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:48.953602 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ee984-ae00-42da-bea2-2a3a90c6fdcb None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:48.953953 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ee984-ae00-42da-bea2-2a3a90c6fdcb None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:48.954294 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ee984-ae00-42da-bea2-2a3a90c6fdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:48.967183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:48.973675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f18ee984-ae00-42da-bea2-2a3a90c6fdcb None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:48.974855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f18ee984-ae00-42da-bea2-2a3a90c6fdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:49.038138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69bbe2-c503-4ae0-a1eb-9e788c31c287 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:49.056789 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 403/804] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:01:49.058081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ee984-ae00-42da-bea2-2a3a90c6fdcb None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:49.078785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b292e-6e83-4b39-b5d8-9d78d59f7e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:49.079406 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b292e-6e83-4b39-b5d8-9d78d59f7e1c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:49.080212 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b292e-6e83-4b39-b5d8-9d78d59f7e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:49.080478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b292e-6e83-4b39-b5d8-9d78d59f7e1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:49.087187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b292e-6e83-4b39-b5d8-9d78d59f7e1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:49.088610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b292e-6e83-4b39-b5d8-9d78d59f7e1c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:49.099614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-1c5b292e-6e83-4b39-b5d8-9d78d59f7e1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:49.099614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:49.099614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:49.099614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:49.099614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:49.099614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:49.099614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:49.099614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:49.099614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:49.099614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:49.099614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:49.099614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:49.099614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:49.099614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:49.099614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:49.100476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:49.101391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:49.101391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:49.101391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:49.101391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:49.101391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:49.101391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:49.101391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:49.101391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:49.101391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 402/805] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:49.102441 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b292e-6e83-4b39-b5d8-9d78d59f7e1c None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:49.133428 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:49.167916 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ee984-ae00-42da-bea2-2a3a90c6fdcb None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:49.198014 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:49.205094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd23ed-5758-4912-bfd8-6dccc76f9ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:49.205596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b292e-6e83-4b39-b5d8-9d78d59f7e1c None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:49.205716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd23ed-5758-4912-bfd8-6dccc76f9ced None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:49.205994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd23ed-5758-4912-bfd8-6dccc76f9ced None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:49.206265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd23ed-5758-4912-bfd8-6dccc76f9ced None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:49.206850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd23ed-5758-4912-bfd8-6dccc76f9ced None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:49.208225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd23ed-5758-4912-bfd8-6dccc76f9ced None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-96bd23ed-5758-4912-bfd8-6dccc76f9ced None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:49.217007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:49.218025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:49.218864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:49.218864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:49.218864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:49.218864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:49.218864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:49.218864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:49.218864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:49.218864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:49.218864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:49.218864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:49.218864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:01:49.218864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:49.218864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 12 12:01:49.218864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:49.218864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 404/806] 10.222.0.27 () {56 vars in 1129 bytes} [Fri Aug 12 12:01:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:49.219645 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd23ed-5758-4912-bfd8-6dccc76f9ced None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:49.246787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20065ffe-91e9-4d5a-aafd-68b31fb01ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:49.247394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20065ffe-91e9-4d5a-aafd-68b31fb01ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:49.247672 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20065ffe-91e9-4d5a-aafd-68b31fb01ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:49.247944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20065ffe-91e9-4d5a-aafd-68b31fb01ca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:49.321268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20065ffe-91e9-4d5a-aafd-68b31fb01ca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:49.322726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20065ffe-91e9-4d5a-aafd-68b31fb01ca6 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:49.323543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd23ed-5758-4912-bfd8-6dccc76f9ced None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:49.351464 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:49.356917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129456d-8d5c-463a-b97e-d566f150a6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:49.357538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129456d-8d5c-463a-b97e-d566f150a6fe None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:49.357820 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129456d-8d5c-463a-b97e-d566f150a6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:49.358092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129456d-8d5c-463a-b97e-d566f150a6fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:49.368769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129456d-8d5c-463a-b97e-d566f150a6fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-1129456d-8d5c-463a-b97e-d566f150a6fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:49.373493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:49.374463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:49.375293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:49.375293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:49.375293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:49.375293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:49.375293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:49.375293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:49.375293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:49.375293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:49.375293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:49.375293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:49.375293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:49.375293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:49.377069 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 405/807] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:01:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:49.377920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129456d-8d5c-463a-b97e-d566f150a6fe None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:49.411844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:49.418934 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 403/808] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:01:49.419727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20065ffe-91e9-4d5a-aafd-68b31fb01ca6 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:49.478817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129456d-8d5c-463a-b97e-d566f150a6fe None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:49.511538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2be4f-ba7f-43c3-bd3c-69ea416a79e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:49.512139 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2be4f-ba7f-43c3-bd3c-69ea416a79e2 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:49.512424 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2be4f-ba7f-43c3-bd3c-69ea416a79e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:49.512729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2be4f-ba7f-43c3-bd3c-69ea416a79e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:49.513424 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2be4f-ba7f-43c3-bd3c-69ea416a79e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:49.513729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2be4f-ba7f-43c3-bd3c-69ea416a79e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:49.515155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2be4f-ba7f-43c3-bd3c-69ea416a79e2 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:49.525101 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20065ffe-91e9-4d5a-aafd-68b31fb01ca6 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:49.527736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 406/809] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:49.528529 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2be4f-ba7f-43c3-bd3c-69ea416a79e2 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:49.563141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb89246-1fb7-4699-b702-c32a1f479540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:49.563739 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb89246-1fb7-4699-b702-c32a1f479540 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:49.564053 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb89246-1fb7-4699-b702-c32a1f479540 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:49.564320 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb89246-1fb7-4699-b702-c32a1f479540 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:49.564976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb89246-1fb7-4699-b702-c32a1f479540 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:49.565282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb89246-1fb7-4699-b702-c32a1f479540 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:49.566639 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb89246-1fb7-4699-b702-c32a1f479540 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:49.570948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 404/810] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Aug 12 12:01:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:49.571785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb89246-1fb7-4699-b702-c32a1f479540 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:49.583579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:49.629950 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:49.632207 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2be4f-ba7f-43c3-bd3c-69ea416a79e2 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:49.666032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a223c-fd60-42b6-8249-f0adcbd5e532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:49.666648 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a223c-fd60-42b6-8249-f0adcbd5e532 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:49.666923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a223c-fd60-42b6-8249-f0adcbd5e532 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:49.667196 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a223c-fd60-42b6-8249-f0adcbd5e532 None admin] PATH_INFO: `/v3/services/network` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:49.667766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a223c-fd60-42b6-8249-f0adcbd5e532 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:49.669177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801a223c-fd60-42b6-8249-f0adcbd5e532 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:49.670182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb89246-1fb7-4699-b702-c32a1f479540 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-801a223c-fd60-42b6-8249-f0adcbd5e532 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:49.678089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:49.679505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:49.680515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:49.680515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:49.680515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:49.680515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:49.680515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:49.680515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:49.680515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:49.680515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:49.680515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:49.680515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:49.680515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:01:49.680515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:49.680515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 12 12:01:49.680515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:49.680515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 407/811] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:49 2022] GET /identity/v3/services/network => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:49.681371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a223c-fd60-42b6-8249-f0adcbd5e532 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:49.709422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec84b73-828d-4cc8-a6ca-52cbd23e78de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:49.710014 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec84b73-828d-4cc8-a6ca-52cbd23e78de None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:49.710287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec84b73-828d-4cc8-a6ca-52cbd23e78de None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:49.710560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec84b73-828d-4cc8-a6ca-52cbd23e78de None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:49.711211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec84b73-828d-4cc8-a6ca-52cbd23e78de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:49.711508 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec84b73-828d-4cc8-a6ca-52cbd23e78de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:49.712948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec84b73-828d-4cc8-a6ca-52cbd23e78de None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:49.721199 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 405/812] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:49.721695 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec84b73-828d-4cc8-a6ca-52cbd23e78de None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:49.781626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a223c-fd60-42b6-8249-f0adcbd5e532 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:49.795054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:49.818850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19ce6f-3f10-4307-bfe1-727d04ddbc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:49.819481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19ce6f-3f10-4307-bfe1-727d04ddbc3e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:49.819861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19ce6f-3f10-4307-bfe1-727d04ddbc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:49.820168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19ce6f-3f10-4307-bfe1-727d04ddbc3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:49.821260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec84b73-828d-4cc8-a6ca-52cbd23e78de None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:49.832017 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19ce6f-3f10-4307-bfe1-727d04ddbc3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:49.833499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19ce6f-3f10-4307-bfe1-727d04ddbc3e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:49.840991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-3d19ce6f-3f10-4307-bfe1-727d04ddbc3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:49.848481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:49.849802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:49.851020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:49.851020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:49.851020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:49.851020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 408/813] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:49.851020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19ce6f-3f10-4307-bfe1-727d04ddbc3e None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:49.859627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee00735-72ff-4099-af72-cb87837cc2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:49.860369 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee00735-72ff-4099-af72-cb87837cc2d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:49.860632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee00735-72ff-4099-af72-cb87837cc2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:49.861027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee00735-72ff-4099-af72-cb87837cc2d8 None admin] PATH_INFO: `/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/ab4e815bbe7f4ff880f4cc633da1ed8c/roles/1416a05bf5ba4de88889c54ff5c58da2` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:49.875821 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee00735-72ff-4099-af72-cb87837cc2d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=20c5570631a34c4ab3868eb1cd96e345, user_id=ab4e815bbe7f4ff880f4cc633da1ed8c, role_id=1416a05bf5ba4de88889c54ff5c58da2)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:49.877860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee00735-72ff-4099-af72-cb87837cc2d8 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:49.942523 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 406/814] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:01:49 2022] PUT /identity/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/ab4e815bbe7f4ff880f4cc633da1ed8c/roles/1416a05bf5ba4de88889c54ff5c58da2 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:01:49.943342 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee00735-72ff-4099-af72-cb87837cc2d8 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:49.960416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19ce6f-3f10-4307-bfe1-727d04ddbc3e None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:50.003262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c74754-f0a1-45da-a4b5-293dad652f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:50.003891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c74754-f0a1-45da-a4b5-293dad652f7a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:50.004174 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c74754-f0a1-45da-a4b5-293dad652f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:50.004449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c74754-f0a1-45da-a4b5-293dad652f7a None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:50.005121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c74754-f0a1-45da-a4b5-293dad652f7a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:50.005827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c74754-f0a1-45da-a4b5-293dad652f7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:50.006858 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c74754-f0a1-45da-a4b5-293dad652f7a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:50.012878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 409/815] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Aug 12 12:01:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:50.013694 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c74754-f0a1-45da-a4b5-293dad652f7a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:50.049372 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:50.066241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee00735-72ff-4099-af72-cb87837cc2d8 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:50.106398 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d109f5b-eb2f-4b97-9183-a3f090924d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:50.106999 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d109f5b-eb2f-4b97-9183-a3f090924d81 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:50.107278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d109f5b-eb2f-4b97-9183-a3f090924d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:50.109316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d109f5b-eb2f-4b97-9183-a3f090924d81 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:50.109316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d109f5b-eb2f-4b97-9183-a3f090924d81 None admin] RBAC: Adding query filter params (name=network) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:50.109316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d109f5b-eb2f-4b97-9183-a3f090924d81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:50.109841 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d109f5b-eb2f-4b97-9183-a3f090924d81 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:50.117072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 407/816] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 12 12:01:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:50.117731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d109f5b-eb2f-4b97-9183-a3f090924d81 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:50.135585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c74754-f0a1-45da-a4b5-293dad652f7a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:50.165361 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:50.179210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6d85a-db27-42b9-b6f2-fa214349476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:50.180205 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6d85a-db27-42b9-b6f2-fa214349476c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:50.180781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6d85a-db27-42b9-b6f2-fa214349476c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:50.181058 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6d85a-db27-42b9-b6f2-fa214349476c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:50.194902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6d85a-db27-42b9-b6f2-fa214349476c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:50.196474 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6d85a-db27-42b9-b6f2-fa214349476c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-c0c6d85a-db27-42b9-b6f2-fa214349476c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:50.211647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:50.212627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:50.213538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:50.213538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:50.213538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:50.213538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 410/817] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:50.213538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6d85a-db27-42b9-b6f2-fa214349476c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:50.237492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d109f5b-eb2f-4b97-9183-a3f090924d81 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:50.273949 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be141ec3-5222-4e4b-bdef-cbe8bd7ff952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:50.274662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be141ec3-5222-4e4b-bdef-cbe8bd7ff952 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:50.275011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be141ec3-5222-4e4b-bdef-cbe8bd7ff952 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:50.275362 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be141ec3-5222-4e4b-bdef-cbe8bd7ff952 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:50.276086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be141ec3-5222-4e4b-bdef-cbe8bd7ff952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:50.276465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be141ec3-5222-4e4b-bdef-cbe8bd7ff952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:50.277929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be141ec3-5222-4e4b-bdef-cbe8bd7ff952 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:50.294928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 408/818] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:50.294928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be141ec3-5222-4e4b-bdef-cbe8bd7ff952 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:50.318689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:50.337778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6d85a-db27-42b9-b6f2-fa214349476c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:50.379798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee6b6f-137d-42ce-8018-ba0031ae55ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:50.379798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee6b6f-137d-42ce-8018-ba0031ae55ab None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:50.379798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee6b6f-137d-42ce-8018-ba0031ae55ab None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:50.379798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee6b6f-137d-42ce-8018-ba0031ae55ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:50.380251 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee6b6f-137d-42ce-8018-ba0031ae55ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:50.380993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee6b6f-137d-42ce-8018-ba0031ae55ab None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:50.409238 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:50.416065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be141ec3-5222-4e4b-bdef-cbe8bd7ff952 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:50.449110 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba2124-5cca-4e8b-a03d-1c8454f10495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:50.449836 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba2124-5cca-4e8b-a03d-1c8454f10495 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:50.450184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba2124-5cca-4e8b-a03d-1c8454f10495 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:50.450544 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba2124-5cca-4e8b-a03d-1c8454f10495 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:50.451246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba2124-5cca-4e8b-a03d-1c8454f10495 None admin] RBAC: Adding query filter params (type=network) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:50.451621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba2124-5cca-4e8b-a03d-1c8454f10495 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:50.453080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba2124-5cca-4e8b-a03d-1c8454f10495 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:50.459627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 409/819] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 12 12:01:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:50.460429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba2124-5cca-4e8b-a03d-1c8454f10495 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:50.492987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 411/820] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:01:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:01:50.494986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee6b6f-137d-42ce-8018-ba0031ae55ab None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:50.616388 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba2124-5cca-4e8b-a03d-1c8454f10495 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:50.625371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee6b6f-137d-42ce-8018-ba0031ae55ab None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:50.665022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e90d7c-bd29-4879-8c08-227e95c23188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:50.665636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e90d7c-bd29-4879-8c08-227e95c23188 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:50.665921 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e90d7c-bd29-4879-8c08-227e95c23188 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:50.666190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e90d7c-bd29-4879-8c08-227e95c23188 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:50.666955 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e90d7c-bd29-4879-8c08-227e95c23188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:50.667151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e90d7c-bd29-4879-8c08-227e95c23188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:50.668504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e90d7c-bd29-4879-8c08-227e95c23188 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:50.680889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aff1fa-23ae-4fac-8d90-fe36b22fd51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:50.681861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aff1fa-23ae-4fac-8d90-fe36b22fd51a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:50.682189 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aff1fa-23ae-4fac-8d90-fe36b22fd51a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:50.682470 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aff1fa-23ae-4fac-8d90-fe36b22fd51a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:50.683536 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 410/821] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:50.684174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e90d7c-bd29-4879-8c08-227e95c23188 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:50.685779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:50.693692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aff1fa-23ae-4fac-8d90-fe36b22fd51a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:50.695431 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aff1fa-23ae-4fac-8d90-fe36b22fd51a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:50.708984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-21aff1fa-23ae-4fac-8d90-fe36b22fd51a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:50.708984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:50.708984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:50.708984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:50.708984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:50.708984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:50.708984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:50.708984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:50.708984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:50.708984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:50.708984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:50.708984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:50.708984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:50.708984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:50.708984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:50.709934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:50.710811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:50.710811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:50.710811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:50.710811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:50.710811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:50.710811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:50.710811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:50.710811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:50.712921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 412/822] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:50.713883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aff1fa-23ae-4fac-8d90-fe36b22fd51a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:50.716920 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:50.808952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e90d7c-bd29-4879-8c08-227e95c23188 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:50.849010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c39178-20ae-4378-9c16-ca485e10cfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:50.849605 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c39178-20ae-4378-9c16-ca485e10cfa2 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:50.849883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c39178-20ae-4378-9c16-ca485e10cfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:50.850151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c39178-20ae-4378-9c16-ca485e10cfa2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:50.850806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c39178-20ae-4378-9c16-ca485e10cfa2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:50.853275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c39178-20ae-4378-9c16-ca485e10cfa2 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:50.855709 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aff1fa-23ae-4fac-8d90-fe36b22fd51a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:50.893008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb10a0a-c5d1-4970-8fbc-f8c938baae05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:50.893745 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb10a0a-c5d1-4970-8fbc-f8c938baae05 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:50.894026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb10a0a-c5d1-4970-8fbc-f8c938baae05 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:50.894385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb10a0a-c5d1-4970-8fbc-f8c938baae05 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:50.902718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb10a0a-c5d1-4970-8fbc-f8c938baae05 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:50.904267 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb10a0a-c5d1-4970-8fbc-f8c938baae05 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-7bb10a0a-c5d1-4970-8fbc-f8c938baae05 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:50.916345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:50.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:50.918796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:50.918796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:50.918796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:50.918796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:50.918796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:50.918796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:50.918796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:50.918796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:50.918796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 12 12:01:50.918796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:50.920716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 413/823] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:01:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:50.921684 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb10a0a-c5d1-4970-8fbc-f8c938baae05 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:50.978178 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 411/824] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:01:50 2022] POST /identity/v3/endpoints => generated 337 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:01:50.979733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c39178-20ae-4378-9c16-ca485e10cfa2 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:50.999491 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:51.091255 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:51.111799 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb10a0a-c5d1-4970-8fbc-f8c938baae05 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:51.128739 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c39178-20ae-4378-9c16-ca485e10cfa2 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:51.167065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897ccc5-6142-405a-8042-f4758f1f645e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:51.168002 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897ccc5-6142-405a-8042-f4758f1f645e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:51.168402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897ccc5-6142-405a-8042-f4758f1f645e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:51.168901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897ccc5-6142-405a-8042-f4758f1f645e None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:51.170186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897ccc5-6142-405a-8042-f4758f1f645e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:51.170186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897ccc5-6142-405a-8042-f4758f1f645e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:51.172049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897ccc5-6142-405a-8042-f4758f1f645e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:51.172483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674175d-c623-4fb1-b987-15b27041ac3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:51.173836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674175d-c623-4fb1-b987-15b27041ac3d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:51.174268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674175d-c623-4fb1-b987-15b27041ac3d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:51.175081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674175d-c623-4fb1-b987-15b27041ac3d None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:51.176122 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674175d-c623-4fb1-b987-15b27041ac3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:51.177437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674175d-c623-4fb1-b987-15b27041ac3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:51.179274 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674175d-c623-4fb1-b987-15b27041ac3d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:51.194966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 414/825] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:51.196672 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 412/826] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Aug 12 12:01:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:51.201223 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674175d-c623-4fb1-b987-15b27041ac3d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:51.205721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897ccc5-6142-405a-8042-f4758f1f645e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:51.317510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897ccc5-6142-405a-8042-f4758f1f645e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:51.323075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674175d-c623-4fb1-b987-15b27041ac3d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:51.338472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:51.347984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:51.356728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba46c2-0756-4cfb-9e1e-7982c89a66a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:51.357873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba46c2-0756-4cfb-9e1e-7982c89a66a6 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:51.358465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba46c2-0756-4cfb-9e1e-7982c89a66a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:51.359001 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba46c2-0756-4cfb-9e1e-7982c89a66a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:51.359894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944e176-b66d-497a-a7d0-a0b80ce79578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:51.361113 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944e176-b66d-497a-a7d0-a0b80ce79578 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:51.361474 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944e176-b66d-497a-a7d0-a0b80ce79578 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:51.361812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944e176-b66d-497a-a7d0-a0b80ce79578 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:51.362777 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944e176-b66d-497a-a7d0-a0b80ce79578 None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=687d0375abd44d2481f988c6df90fad1) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:51.363036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944e176-b66d-497a-a7d0-a0b80ce79578 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:51.364556 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944e176-b66d-497a-a7d0-a0b80ce79578 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:51.370988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 415/827] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:51 2022] GET /identity/v3/role_assignments?user.id=687d0375abd44d2481f988c6df90fad1&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=906e3a8a6c7540a38f629f2567133347 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:51.372440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba46c2-0756-4cfb-9e1e-7982c89a66a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:51.373936 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba46c2-0756-4cfb-9e1e-7982c89a66a6 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-18ba46c2-0756-4cfb-9e1e-7982c89a66a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:51.397991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:51.398968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:51.399825 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:51.399825 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:51.399825 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:51.402175 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 413/828] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:51.409267 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944e176-b66d-497a-a7d0-a0b80ce79578 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:51.525241 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944e176-b66d-497a-a7d0-a0b80ce79578 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:51.563091 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3d518-f87c-46af-bff4-f5d7abe14d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:51.564623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3d518-f87c-46af-bff4-f5d7abe14d78 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:51.565120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3d518-f87c-46af-bff4-f5d7abe14d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:51.565612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3d518-f87c-46af-bff4-f5d7abe14d78 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:51.566477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3d518-f87c-46af-bff4-f5d7abe14d78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:51.566984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3d518-f87c-46af-bff4-f5d7abe14d78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:51.568542 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3d518-f87c-46af-bff4-f5d7abe14d78 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:51.581598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 416/829] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:51.586821 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba46c2-0756-4cfb-9e1e-7982c89a66a6 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:51.694491 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba46c2-0756-4cfb-9e1e-7982c89a66a6 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:51.730138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce286d-a8c9-4cb9-977c-26ba9a019437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:51.730767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce286d-a8c9-4cb9-977c-26ba9a019437 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:51.731051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce286d-a8c9-4cb9-977c-26ba9a019437 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:51.731329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce286d-a8c9-4cb9-977c-26ba9a019437 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:51.738319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce286d-a8c9-4cb9-977c-26ba9a019437 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:51.739771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce286d-a8c9-4cb9-977c-26ba9a019437 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:51.752543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-fbce286d-a8c9-4cb9-977c-26ba9a019437 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:51.752543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:51.752543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:51.752543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:51.752543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:51.752543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:51.752543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:51.752543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:51.752543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:51.752543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:51.752543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:51.752543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:51.752543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:51.752543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:51.752543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:51.753571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:51.755027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:51.755027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:51.755027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:51.755027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:51.755027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:51.755027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:51.755027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:51.755027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:51.755027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 414/830] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:51.759794 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3d518-f87c-46af-bff4-f5d7abe14d78 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:51.809388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:51.866463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3d518-f87c-46af-bff4-f5d7abe14d78 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:51.901802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84292763-b8e2-4bb0-b57e-ac78611d1681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:51.902647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84292763-b8e2-4bb0-b57e-ac78611d1681 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:51.903072 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84292763-b8e2-4bb0-b57e-ac78611d1681 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:51.903515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84292763-b8e2-4bb0-b57e-ac78611d1681 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:51.904345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84292763-b8e2-4bb0-b57e-ac78611d1681 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:51.904830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84292763-b8e2-4bb0-b57e-ac78611d1681 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:51.906400 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84292763-b8e2-4bb0-b57e-ac78611d1681 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:51.919200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 417/831] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:51.925945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce286d-a8c9-4cb9-977c-26ba9a019437 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:51.971407 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:51.985258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411804fa-7f06-4561-aba5-e0357d32c653 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:51.986827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411804fa-7f06-4561-aba5-e0357d32c653 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:51.990528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411804fa-7f06-4561-aba5-e0357d32c653 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:51.990528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 418/832] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:01:51.995391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31549abe-4e05-4705-b953-29420362b8b2 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:51.995678 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31549abe-4e05-4705-b953-29420362b8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:51.995952 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31549abe-4e05-4705-b953-29420362b8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:52.015218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-31549abe-4e05-4705-b953-29420362b8b2 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:52.016441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-31549abe-4e05-4705-b953-29420362b8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:52.040905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce286d-a8c9-4cb9-977c-26ba9a019437 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:52.077916 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53365cbd-61d0-44a3-99ca-cdba6a8b1dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:52.078911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53365cbd-61d0-44a3-99ca-cdba6a8b1dbf None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:52.079447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53365cbd-61d0-44a3-99ca-cdba6a8b1dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:52.079931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53365cbd-61d0-44a3-99ca-cdba6a8b1dbf None admin] PATH_INFO: `/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/5123401076a84ffcab49b00fee65d14d/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:52.082192 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53365cbd-61d0-44a3-99ca-cdba6a8b1dbf None admin] RBAC: Authorizing `identity:create_grant(project_id=20c5570631a34c4ab3868eb1cd96e345, user_id=5123401076a84ffcab49b00fee65d14d, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:52.083817 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53365cbd-61d0-44a3-99ca-cdba6a8b1dbf None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:52.088846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 419/833] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:01:52.159589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 415/834] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:01:51 2022] PUT /identity/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/5123401076a84ffcab49b00fee65d14d/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:01:52.373600 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31549abe-4e05-4705-b953-29420362b8b2 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:52.414288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:52.515073 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31549abe-4e05-4705-b953-29420362b8b2 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:52.571965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306452d-c3b2-4b1b-899c-1245ea417fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:52.572606 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306452d-c3b2-4b1b-899c-1245ea417fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:52.572909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306452d-c3b2-4b1b-899c-1245ea417fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:52.575165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306452d-c3b2-4b1b-899c-1245ea417fa5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:52.594362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306452d-c3b2-4b1b-899c-1245ea417fa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-f306452d-c3b2-4b1b-899c-1245ea417fa5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:52.599117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:52.600047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:52.600887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:52.600887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:52.600887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:52.600887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:52.600887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:52.600887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:52.600887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:52.600887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:52.600887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:52.600887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:52.600887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:01:52.600887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:52.606859 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 420/835] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:01:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:52.609714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53365cbd-61d0-44a3-99ca-cdba6a8b1dbf None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:52.685217 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8575a-8649-4df3-848b-fd7cc6e21d96 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:52.685860 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8575a-8649-4df3-848b-fd7cc6e21d96 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:52.686361 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8575a-8649-4df3-848b-fd7cc6e21d96 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:52.688149 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 421/836] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:52.693747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778edd23-175d-4a78-ac76-a4d51395286d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:52.694303 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778edd23-175d-4a78-ac76-a4d51395286d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:52.694781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778edd23-175d-4a78-ac76-a4d51395286d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:52.728722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-778edd23-175d-4a78-ac76-a4d51395286d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:52.730245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-778edd23-175d-4a78-ac76-a4d51395286d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:52.764439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:52.771173 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53365cbd-61d0-44a3-99ca-cdba6a8b1dbf None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:52.837234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e8b64-bf1d-465e-9f0c-b85f5d6a1410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:52.838191 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e8b64-bf1d-465e-9f0c-b85f5d6a1410 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:52.838682 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e8b64-bf1d-465e-9f0c-b85f5d6a1410 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:52.839150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e8b64-bf1d-465e-9f0c-b85f5d6a1410 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:52.840114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e8b64-bf1d-465e-9f0c-b85f5d6a1410 None admin] RBAC: Adding query filter params (name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:52.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e8b64-bf1d-465e-9f0c-b85f5d6a1410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:52.842225 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e8b64-bf1d-465e-9f0c-b85f5d6a1410 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:52.852470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 416/837] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:01:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:52.859294 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e8b64-bf1d-465e-9f0c-b85f5d6a1410 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:52.871479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 422/838] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:01:52.972674 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e8b64-bf1d-465e-9f0c-b85f5d6a1410 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:53.008545 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe1685-f960-4b8c-8111-89cec87c1c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:53.009591 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1685-f960-4b8c-8111-89cec87c1c7a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:53.010052 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1685-f960-4b8c-8111-89cec87c1c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:53.010498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1685-f960-4b8c-8111-89cec87c1c7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:53.021689 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe1685-f960-4b8c-8111-89cec87c1c7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:53.023395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe1685-f960-4b8c-8111-89cec87c1c7a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-e8fe1685-f960-4b8c-8111-89cec87c1c7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:53.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:53.046832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:53.047727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:53.047727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:53.047727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:53.047727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 417/839] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:53.050466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778edd23-175d-4a78-ac76-a4d51395286d None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:53.071836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:53.160305 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778edd23-175d-4a78-ac76-a4d51395286d None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:53.181403 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe1685-f960-4b8c-8111-89cec87c1c7a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:53.198781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:53.198971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55d245-b730-4c0f-b173-676434d14c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:53.199486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55d245-b730-4c0f-b173-676434d14c88 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:53.200126 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55d245-b730-4c0f-b173-676434d14c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:53.200126 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55d245-b730-4c0f-b173-676434d14c88 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:53.200866 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55d245-b730-4c0f-b173-676434d14c88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:53.200988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55d245-b730-4c0f-b173-676434d14c88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:53.202363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55d245-b730-4c0f-b173-676434d14c88 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:53.220786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 423/840] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:53.222194 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55d245-b730-4c0f-b173-676434d14c88 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:53.323109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe1685-f960-4b8c-8111-89cec87c1c7a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:53.327953 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:53.353311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55d245-b730-4c0f-b173-676434d14c88 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:53.363494 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e2795-227e-4d29-8651-705b098dda7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:53.364431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e2795-227e-4d29-8651-705b098dda7a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:53.364942 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e2795-227e-4d29-8651-705b098dda7a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:53.365454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e2795-227e-4d29-8651-705b098dda7a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:53.366308 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e2795-227e-4d29-8651-705b098dda7a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:53.367869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e2795-227e-4d29-8651-705b098dda7a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-298e2795-227e-4d29-8651-705b098dda7a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:53.378976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:53.379954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:53.380886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:53.380886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:53.380886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:53.380886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:53.380886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:53.380886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:53.380886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:53.380886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:53.380886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:53.380886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:53.380886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:01:53.380886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:53.380886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 12:01:53.380886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:53.388475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 418/841] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:01:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:53.389578 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e2795-227e-4d29-8651-705b098dda7a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:53.392273 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44452d6-1d20-4fc1-a528-dac4f5c39df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:53.393281 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44452d6-1d20-4fc1-a528-dac4f5c39df1 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:53.393570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44452d6-1d20-4fc1-a528-dac4f5c39df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:53.393847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44452d6-1d20-4fc1-a528-dac4f5c39df1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:53.401467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44452d6-1d20-4fc1-a528-dac4f5c39df1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:53.402733 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44452d6-1d20-4fc1-a528-dac4f5c39df1 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-a44452d6-1d20-4fc1-a528-dac4f5c39df1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:53.415452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:53.416390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:53.417349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:53.417349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:53.417349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:53.417349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:53.417349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:53.417349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:53.417349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:53.417349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:53.417349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 12:01:53.417349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:53.420791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 424/842] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 12 12:01:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:53.422517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44452d6-1d20-4fc1-a528-dac4f5c39df1 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:53.484450 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:53.509138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e2795-227e-4d29-8651-705b098dda7a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:53.528243 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44452d6-1d20-4fc1-a528-dac4f5c39df1 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:53.547057 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bd10e-99fa-471f-ac70-9c41cfeb8622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:53.547656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bd10e-99fa-471f-ac70-9c41cfeb8622 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:53.548540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bd10e-99fa-471f-ac70-9c41cfeb8622 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:53.548838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bd10e-99fa-471f-ac70-9c41cfeb8622 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:53.549470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bd10e-99fa-471f-ac70-9c41cfeb8622 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:53.549831 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bd10e-99fa-471f-ac70-9c41cfeb8622 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:53.551227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bd10e-99fa-471f-ac70-9c41cfeb8622 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:53.556827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 419/843] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:01:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:53.557147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bab7c4-3e5d-40a1-8ff7-7c8d107cf7af None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:53.557368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bab7c4-3e5d-40a1-8ff7-7c8d107cf7af None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:53.557648 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bab7c4-3e5d-40a1-8ff7-7c8d107cf7af None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:53.559097 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 420/844] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:53.564607 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32d415-f9e6-4887-b9bd-2583729832ac None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:53.564919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32d415-f9e6-4887-b9bd-2583729832ac None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:53.565215 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32d415-f9e6-4887-b9bd-2583729832ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:53.567901 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b858c-7312-4799-a340-093a1405599a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:53.568895 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b858c-7312-4799-a340-093a1405599a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:53.569328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b858c-7312-4799-a340-093a1405599a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:53.569838 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b858c-7312-4799-a340-093a1405599a None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:53.570721 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b858c-7312-4799-a340-093a1405599a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:53.571304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b858c-7312-4799-a340-093a1405599a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:53.573043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b858c-7312-4799-a340-093a1405599a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:53.583004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:53.583728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-1d32d415-f9e6-4887-b9bd-2583729832ac None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:53.585025 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1d32d415-f9e6-4887-b9bd-2583729832ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:53.598207 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 425/845] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Aug 12 12:01:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:53.599286 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b858c-7312-4799-a340-093a1405599a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:53.652621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 421/846] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:01:53.653585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32d415-f9e6-4887-b9bd-2583729832ac None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:53.723032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b858c-7312-4799-a340-093a1405599a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:53.735162 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:53.768618 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32d415-f9e6-4887-b9bd-2583729832ac None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:53.769786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da86d64-8959-4211-bba5-b69c83b26957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:53.770471 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da86d64-8959-4211-bba5-b69c83b26957 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:53.770981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da86d64-8959-4211-bba5-b69c83b26957 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:53.771336 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da86d64-8959-4211-bba5-b69c83b26957 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:53.772727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da86d64-8959-4211-bba5-b69c83b26957 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:53.777034 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da86d64-8959-4211-bba5-b69c83b26957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:53.778943 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da86d64-8959-4211-bba5-b69c83b26957 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:53.785902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 426/847] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:01:53 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:53.813323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f9d79-2924-4538-ae46-fb9e7c8ea5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:53.815202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f9d79-2924-4538-ae46-fb9e7c8ea5ac None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:53.815664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f9d79-2924-4538-ae46-fb9e7c8ea5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:53.816089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f9d79-2924-4538-ae46-fb9e7c8ea5ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:53.897987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f9d79-2924-4538-ae46-fb9e7c8ea5ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:53.899720 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f9d79-2924-4538-ae46-fb9e7c8ea5ac None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:53.963096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da86d64-8959-4211-bba5-b69c83b26957 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:53.989030 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:54.022436 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 422/848] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:53 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:01:54.028875 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f9d79-2924-4538-ae46-fb9e7c8ea5ac None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:54.043162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:54.096942 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da86d64-8959-4211-bba5-b69c83b26957 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:54.138810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd0b31-58e0-4a7c-aeae-c6bc4217ea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:54.139532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd0b31-58e0-4a7c-aeae-c6bc4217ea72 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:54.139885 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd0b31-58e0-4a7c-aeae-c6bc4217ea72 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:54.140229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd0b31-58e0-4a7c-aeae-c6bc4217ea72 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:54.140929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd0b31-58e0-4a7c-aeae-c6bc4217ea72 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:54.143456 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd0b31-58e0-4a7c-aeae-c6bc4217ea72 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-2ecd0b31-58e0-4a7c-aeae-c6bc4217ea72 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:54.152261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:54.153235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:54.154114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:54.154114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:54.154114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:54.154114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:54.154114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:54.154114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:54.154114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:54.154114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:54.154114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:54.154114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:54.154114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:01:54.154114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:54.154114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 12:01:54.154114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:54.157058 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 427/849] 10.222.0.27 () {56 vars in 1132 bytes} [Fri Aug 12 12:01:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:54.159734 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f9d79-2924-4538-ae46-fb9e7c8ea5ac None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:54.160185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd0b31-58e0-4a7c-aeae-c6bc4217ea72 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:54.196516 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5aed1b-42d0-4a2b-aeee-958ad0234b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:54.197429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5aed1b-42d0-4a2b-aeee-958ad0234b6d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:54.197708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5aed1b-42d0-4a2b-aeee-958ad0234b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:54.197978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5aed1b-42d0-4a2b-aeee-958ad0234b6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:54.208606 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5aed1b-42d0-4a2b-aeee-958ad0234b6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:54.210075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5aed1b-42d0-4a2b-aeee-958ad0234b6d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-0a5aed1b-42d0-4a2b-aeee-958ad0234b6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:54.224432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:54.225429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:54.226286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:54.226286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:54.226286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:54.230276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 423/850] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:54.231277 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9acc30-e460-45c9-96b6-bfa7849f6972 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:54.231566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9acc30-e460-45c9-96b6-bfa7849f6972 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:54.231839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9acc30-e460-45c9-96b6-bfa7849f6972 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:54.236690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 424/851] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:54.236690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9acc30-e460-45c9-96b6-bfa7849f6972 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:54.262332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:54.275589 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd0b31-58e0-4a7c-aeae-c6bc4217ea72 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:54.301204 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:54.312912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e13f97-fb0c-4d9f-b3f5-35b3cfb3263c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:54.314064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e13f97-fb0c-4d9f-b3f5-35b3cfb3263c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:54.314525 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e13f97-fb0c-4d9f-b3f5-35b3cfb3263c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:54.314823 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e13f97-fb0c-4d9f-b3f5-35b3cfb3263c None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:54.315430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e13f97-fb0c-4d9f-b3f5-35b3cfb3263c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:54.315732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e13f97-fb0c-4d9f-b3f5-35b3cfb3263c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:54.317143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e13f97-fb0c-4d9f-b3f5-35b3cfb3263c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:54.321647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 428/852] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 12 12:01:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:54.322887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd641c-2d40-4386-8e81-8ffb83de61e5 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:54.323173 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd641c-2d40-4386-8e81-8ffb83de61e5 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:54.323469 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd641c-2d40-4386-8e81-8ffb83de61e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:54.343041 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9ddd641c-2d40-4386-8e81-8ffb83de61e5 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:54.344219 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9ddd641c-2d40-4386-8e81-8ffb83de61e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:54.350507 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9acc30-e460-45c9-96b6-bfa7849f6972 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:54.389039 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f106c-96a3-46c2-a43a-709556a46130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:54.389736 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f106c-96a3-46c2-a43a-709556a46130 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:54.390019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f106c-96a3-46c2-a43a-709556a46130 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:54.390286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f106c-96a3-46c2-a43a-709556a46130 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:54.391013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f106c-96a3-46c2-a43a-709556a46130 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:54.391224 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f106c-96a3-46c2-a43a-709556a46130 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:54.392762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f106c-96a3-46c2-a43a-709556a46130 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:54.410655 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 425/853] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:54.410655 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f106c-96a3-46c2-a43a-709556a46130 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:54.425499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 429/854] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:01:54.425499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd641c-2d40-4386-8e81-8ffb83de61e5 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:54.526509 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:54.541403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd641c-2d40-4386-8e81-8ffb83de61e5 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:54.548069 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f106c-96a3-46c2-a43a-709556a46130 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:54.560870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:54.578653 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d972c-1752-4250-aaa7-14956fcc764c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:54.579948 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d972c-1752-4250-aaa7-14956fcc764c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:54.580234 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d972c-1752-4250-aaa7-14956fcc764c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:54.580503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d972c-1752-4250-aaa7-14956fcc764c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:54.587609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d972c-1752-4250-aaa7-14956fcc764c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:54.589091 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d972c-1752-4250-aaa7-14956fcc764c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:54.591202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56a8e7-49fc-4cb7-b55e-79888adac456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:54.591992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56a8e7-49fc-4cb7-b55e-79888adac456 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:54.592435 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56a8e7-49fc-4cb7-b55e-79888adac456 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:54.592896 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56a8e7-49fc-4cb7-b55e-79888adac456 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:54.593713 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56a8e7-49fc-4cb7-b55e-79888adac456 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:54.594173 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56a8e7-49fc-4cb7-b55e-79888adac456 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:54.595930 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56a8e7-49fc-4cb7-b55e-79888adac456 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:54.600315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-1b8d972c-1752-4250-aaa7-14956fcc764c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:54.600315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:54.600315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:54.600315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:54.600315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:54.600315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:54.600315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:54.600315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:54.600315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:54.600315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:54.600315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:54.600315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:54.600315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:54.600315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:54.600315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:54.606978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:54.608088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:54.608088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:54.608088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:54.608088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:54.608088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:54.608088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:54.608088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:54.608088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:54.608548 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 426/855] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 12 12:01:54 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:54.608623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 430/856] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:54.610556 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56a8e7-49fc-4cb7-b55e-79888adac456 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:54.703969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d972c-1752-4250-aaa7-14956fcc764c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:54.730169 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56a8e7-49fc-4cb7-b55e-79888adac456 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:54.771751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f68be-968e-4cb7-9245-499f0e80bd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:54.772778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f68be-968e-4cb7-9245-499f0e80bd34 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:54.773328 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f68be-968e-4cb7-9245-499f0e80bd34 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:54.773746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f68be-968e-4cb7-9245-499f0e80bd34 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:54.774582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f68be-968e-4cb7-9245-499f0e80bd34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:54.775054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f68be-968e-4cb7-9245-499f0e80bd34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:54.776771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f68be-968e-4cb7-9245-499f0e80bd34 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:54.789811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 427/857] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:54.796024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f68be-968e-4cb7-9245-499f0e80bd34 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:54.813719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:54.832965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d972c-1752-4250-aaa7-14956fcc764c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:54.874940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf85f4-f807-4950-9361-ee25258ed5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:54.875570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf85f4-f807-4950-9361-ee25258ed5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:54.875979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf85f4-f807-4950-9361-ee25258ed5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:54.876293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf85f4-f807-4950-9361-ee25258ed5f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:54.887487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf85f4-f807-4950-9361-ee25258ed5f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-1fdf85f4-f807-4950-9361-ee25258ed5f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:54.893473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:54.894434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:54.895255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:54.895255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:54.895255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:54.895255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:54.895255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:54.895255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:54.895255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:54.895255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:54.895255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:54.895255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:54.895255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:01:54.895255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:54.899485 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 431/858] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:01:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:54.900839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf85f4-f807-4950-9361-ee25258ed5f7 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:54.945170 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f68be-968e-4cb7-9245-499f0e80bd34 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:54.950864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:54.989134 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af862e05-b824-4186-92d2-a6e4b4a5adf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:54.989812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af862e05-b824-4186-92d2-a6e4b4a5adf9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:54.990284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af862e05-b824-4186-92d2-a6e4b4a5adf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:54.990655 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af862e05-b824-4186-92d2-a6e4b4a5adf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:54.991943 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af862e05-b824-4186-92d2-a6e4b4a5adf9 None admin] RBAC: Adding query filter params (role.id=1416a05bf5ba4de88889c54ff5c58da2, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=ab4e815bbe7f4ff880f4cc633da1ed8c) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:54.991943 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af862e05-b824-4186-92d2-a6e4b4a5adf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:54.993614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af862e05-b824-4186-92d2-a6e4b4a5adf9 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:55.000685 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 428/859] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:54 2022] GET /identity/v3/role_assignments?user.id=ab4e815bbe7f4ff880f4cc633da1ed8c&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=1416a05bf5ba4de88889c54ff5c58da2 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:01:55.036147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf85f4-f807-4950-9361-ee25258ed5f7 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:55.078415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c155d1f-1850-4016-a2e1-ff99615078d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:55.079096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c155d1f-1850-4016-a2e1-ff99615078d7 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:55.079446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c155d1f-1850-4016-a2e1-ff99615078d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:55.079918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c155d1f-1850-4016-a2e1-ff99615078d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:55.080607 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c155d1f-1850-4016-a2e1-ff99615078d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:55.080950 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c155d1f-1850-4016-a2e1-ff99615078d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:55.082358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c155d1f-1850-4016-a2e1-ff99615078d7 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:55.092521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 432/860] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:01:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:55.098611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af862e05-b824-4186-92d2-a6e4b4a5adf9 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:55.176348 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:55.208621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af862e05-b824-4186-92d2-a6e4b4a5adf9 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:55.244566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38904089-adbe-4d60-bde0-2cf3c13fd72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:55.245155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38904089-adbe-4d60-bde0-2cf3c13fd72e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:55.245437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38904089-adbe-4d60-bde0-2cf3c13fd72e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:55.245779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38904089-adbe-4d60-bde0-2cf3c13fd72e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:55.257854 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38904089-adbe-4d60-bde0-2cf3c13fd72e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:55.259313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38904089-adbe-4d60-bde0-2cf3c13fd72e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-38904089-adbe-4d60-bde0-2cf3c13fd72e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:55.274971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:55.275929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:55.277168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:55.277168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:55.277168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:55.277168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 429/861] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:55.283316 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c155d1f-1850-4016-a2e1-ff99615078d7 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:55.389155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c155d1f-1850-4016-a2e1-ff99615078d7 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:55.392822 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:55.425479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9e9d8-9ca0-4629-9144-c9d9e09f7750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:55.426094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9e9d8-9ca0-4629-9144-c9d9e09f7750 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:55.426372 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9e9d8-9ca0-4629-9144-c9d9e09f7750 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:55.426649 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9e9d8-9ca0-4629-9144-c9d9e09f7750 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:55.427360 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9e9d8-9ca0-4629-9144-c9d9e09f7750 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:55.427668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9e9d8-9ca0-4629-9144-c9d9e09f7750 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:55.429385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9e9d8-9ca0-4629-9144-c9d9e09f7750 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:55.442486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 433/862] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:55.448067 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38904089-adbe-4d60-bde0-2cf3c13fd72e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:55.555810 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38904089-adbe-4d60-bde0-2cf3c13fd72e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:55.581870 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:55.595182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d454638-9297-4af1-8ec9-9735e52459d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:55.596120 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d454638-9297-4af1-8ec9-9735e52459d7 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:55.596560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d454638-9297-4af1-8ec9-9735e52459d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:55.597017 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d454638-9297-4af1-8ec9-9735e52459d7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:55.605273 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d454638-9297-4af1-8ec9-9735e52459d7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:55.606717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d454638-9297-4af1-8ec9-9735e52459d7 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-0d454638-9297-4af1-8ec9-9735e52459d7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:55.618460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:55.619637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:55.620767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:55.620767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:55.620767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:55.620767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:55.620767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:55.620767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:55.620767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:55.620767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:55.620767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 12 12:01:55.620767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:55.620767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 430/863] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:01:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:55.631481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9e9d8-9ca0-4629-9144-c9d9e09f7750 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:55.738837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9e9d8-9ca0-4629-9144-c9d9e09f7750 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:55.774613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cfb39-56c3-4812-8616-271318daa844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:55.775559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cfb39-56c3-4812-8616-271318daa844 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:55.776062 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cfb39-56c3-4812-8616-271318daa844 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:55.776557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cfb39-56c3-4812-8616-271318daa844 None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:55.777519 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934cfb39-56c3-4812-8616-271318daa844 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:55.778041 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934cfb39-56c3-4812-8616-271318daa844 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:55.779687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934cfb39-56c3-4812-8616-271318daa844 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:55.790403 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5d978-5fc6-461d-a5ef-fff03110cdbc None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:55.790692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5d978-5fc6-461d-a5ef-fff03110cdbc None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:55.790961 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5d978-5fc6-461d-a5ef-fff03110cdbc None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:55.792547 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 431/864] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:55.798171 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827a9f0-04f0-4c33-94dc-f9e287d10f0b None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:55.798496 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827a9f0-04f0-4c33-94dc-f9e287d10f0b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:55.798954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827a9f0-04f0-4c33-94dc-f9e287d10f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:55.804810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 434/865] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Aug 12 12:01:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:55.807346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cfb39-56c3-4812-8616-271318daa844 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:55.824402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-1827a9f0-04f0-4c33-94dc-f9e287d10f0b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:55.825502 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1827a9f0-04f0-4c33-94dc-f9e287d10f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:55.834890 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:55.914444 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 432/866] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:01:55.932565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cfb39-56c3-4812-8616-271318daa844 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:55.968823 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775c4bb-02d6-483b-8e34-56036d6c29fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:55.970234 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775c4bb-02d6-483b-8e34-56036d6c29fc None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:55.970778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775c4bb-02d6-483b-8e34-56036d6c29fc None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:55.971248 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775c4bb-02d6-483b-8e34-56036d6c29fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:56.043194 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775c4bb-02d6-483b-8e34-56036d6c29fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:56.045086 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775c4bb-02d6-483b-8e34-56036d6c29fc None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:56.064394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:56.151667 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 435/867] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:55 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:01:56.158148 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827a9f0-04f0-4c33-94dc-f9e287d10f0b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:56.224750 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775c4bb-02d6-483b-8e34-56036d6c29fc None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:56.287946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827a9f0-04f0-4c33-94dc-f9e287d10f0b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:56.309683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:56.338249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143312d3-89bf-444f-ad76-e08b956636d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:56.339078 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775c4bb-02d6-483b-8e34-56036d6c29fc None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:56.339786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143312d3-89bf-444f-ad76-e08b956636d0 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:56.340267 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143312d3-89bf-444f-ad76-e08b956636d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:56.341290 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143312d3-89bf-444f-ad76-e08b956636d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:56.355705 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143312d3-89bf-444f-ad76-e08b956636d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:56.358426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143312d3-89bf-444f-ad76-e08b956636d0 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:56.361582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-143312d3-89bf-444f-ad76-e08b956636d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:56.378985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:56.385514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:56.387020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:56.387020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:56.387020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:56.387020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 433/868] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:56.387020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143312d3-89bf-444f-ad76-e08b956636d0 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:56.396626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f672b-07fc-4b1e-967a-a64d2b87e4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:56.398019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f672b-07fc-4b1e-967a-a64d2b87e4e7 None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:56.398540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f672b-07fc-4b1e-967a-a64d2b87e4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:56.399022 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f672b-07fc-4b1e-967a-a64d2b87e4e7 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:56.399919 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f672b-07fc-4b1e-967a-a64d2b87e4e7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:56.401899 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f672b-07fc-4b1e-967a-a64d2b87e4e7 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:56.481085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 436/869] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 12 12:01:56 2022] POST /identity/v3/services => generated 251 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:01:56.518467 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143312d3-89bf-444f-ad76-e08b956636d0 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:56.570673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6f893-b471-4a01-b273-2bc07664f9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:56.571911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6f893-b471-4a01-b273-2bc07664f9db None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:56.572380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6f893-b471-4a01-b273-2bc07664f9db None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:56.572877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6f893-b471-4a01-b273-2bc07664f9db None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:56.573992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6f893-b471-4a01-b273-2bc07664f9db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:56.574503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6f893-b471-4a01-b273-2bc07664f9db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:56.576956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6f893-b471-4a01-b273-2bc07664f9db None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:56.592938 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 434/870] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:56.599454 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f672b-07fc-4b1e-967a-a64d2b87e4e7 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:56.641568 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7cce8-cfa3-416e-a6f8-e519acd2471e None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:56.641954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7cce8-cfa3-416e-a6f8-e519acd2471e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:56.642124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7cce8-cfa3-416e-a6f8-e519acd2471e None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:56.644856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 435/871] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:01:56.659602 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9460a6-b1b0-477e-a22d-7f2f1dd0d032 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:56.660270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9460a6-b1b0-477e-a22d-7f2f1dd0d032 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:56.660563 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9460a6-b1b0-477e-a22d-7f2f1dd0d032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:56.685300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-7a9460a6-b1b0-477e-a22d-7f2f1dd0d032 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:56.686514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-7a9460a6-b1b0-477e-a22d-7f2f1dd0d032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:56.709717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:56.739399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f672b-07fc-4b1e-967a-a64d2b87e4e7 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:56.772417 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 436/872] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:56 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:01:56.786688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83689c2-353f-4c61-a40c-4615422345d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:56.787319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83689c2-353f-4c61-a40c-4615422345d1 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:56.787598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83689c2-353f-4c61-a40c-4615422345d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:56.788094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83689c2-353f-4c61-a40c-4615422345d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:56.795633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83689c2-353f-4c61-a40c-4615422345d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:56.797683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83689c2-353f-4c61-a40c-4615422345d1 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:56.814971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-f83689c2-353f-4c61-a40c-4615422345d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:56.814971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:56.814971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:56.814971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:56.814971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:56.814971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:56.814971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:56.814971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:56.814971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:56.814971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:56.814971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:56.814971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:56.814971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:56.814971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:56.814971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:56.815864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:56.817036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:56.817036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:56.817036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:56.817036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:56.817036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:56.817036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:56.817036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:56.817036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:56.817036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 437/873] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:56.822592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9460a6-b1b0-477e-a22d-7f2f1dd0d032 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:56.944792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9460a6-b1b0-477e-a22d-7f2f1dd0d032 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:56.987531 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e55e88-b5b2-4b45-b67c-8c04a804bfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:56.988209 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e55e88-b5b2-4b45-b67c-8c04a804bfb3 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:56.988485 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e55e88-b5b2-4b45-b67c-8c04a804bfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:56.988776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e55e88-b5b2-4b45-b67c-8c04a804bfb3 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:56.989517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e55e88-b5b2-4b45-b67c-8c04a804bfb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:56.989820 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e55e88-b5b2-4b45-b67c-8c04a804bfb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:56.991250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e55e88-b5b2-4b45-b67c-8c04a804bfb3 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:57.005229 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 437/874] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:57.011297 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83689c2-353f-4c61-a40c-4615422345d1 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:57.046522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e55e88-b5b2-4b45-b67c-8c04a804bfb3 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:57.064963 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:57.065905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:57.157125 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83689c2-353f-4c61-a40c-4615422345d1 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:57.166965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e55e88-b5b2-4b45-b67c-8c04a804bfb3 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:57.195312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c796c62-6dba-450b-9e07-de4dcf71d12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:57.195836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c796c62-6dba-450b-9e07-de4dcf71d12f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:57.196121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c796c62-6dba-450b-9e07-de4dcf71d12f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:57.196393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c796c62-6dba-450b-9e07-de4dcf71d12f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:57.197146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c796c62-6dba-450b-9e07-de4dcf71d12f None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=5123401076a84ffcab49b00fee65d14d) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:57.197455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c796c62-6dba-450b-9e07-de4dcf71d12f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:57.198876 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c796c62-6dba-450b-9e07-de4dcf71d12f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:57.203913 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97fc32-deae-4d8d-9939-d953bab1858b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:57.204766 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97fc32-deae-4d8d-9939-d953bab1858b None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:57.204877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 438/875] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:57 2022] GET /identity/v3/role_assignments?user.id=5123401076a84ffcab49b00fee65d14d&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=906e3a8a6c7540a38f629f2567133347 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:57.205285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97fc32-deae-4d8d-9939-d953bab1858b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:57.205690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97fc32-deae-4d8d-9939-d953bab1858b None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:57.205984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026783e6-7209-412c-99bf-432fe95083f6 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:57.206266 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026783e6-7209-412c-99bf-432fe95083f6 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:57.206546 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026783e6-7209-412c-99bf-432fe95083f6 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:57.206904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97fc32-deae-4d8d-9939-d953bab1858b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:57.207986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 439/876] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:57.208576 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97fc32-deae-4d8d-9939-d953bab1858b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:57.214464 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f24bec-f7a4-4b8b-8cd7-6c80b4007197 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:57.214748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f24bec-f7a4-4b8b-8cd7-6c80b4007197 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:57.215025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f24bec-f7a4-4b8b-8cd7-6c80b4007197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:57.235719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-49f24bec-f7a4-4b8b-8cd7-6c80b4007197 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:57.237093 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-49f24bec-f7a4-4b8b-8cd7-6c80b4007197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:57.314656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 438/877] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 12 12:01:57 2022] POST /identity/v3/services => generated 269 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:01:57.334809 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 440/878] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:01:57.608199 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97fc32-deae-4d8d-9939-d953bab1858b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:57.684311 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:57.715339 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97fc32-deae-4d8d-9939-d953bab1858b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:57.750804 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb4a95-98cb-4dd0-937b-d9f8a9902f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:57.751435 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb4a95-98cb-4dd0-937b-d9f8a9902f0b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:57.751844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb4a95-98cb-4dd0-937b-d9f8a9902f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:57.752120 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb4a95-98cb-4dd0-937b-d9f8a9902f0b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:57.762877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb4a95-98cb-4dd0-937b-d9f8a9902f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-9feb4a95-98cb-4dd0-937b-d9f8a9902f0b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:01:57.769088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:57.769956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:57.770770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:57.770770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:57.770770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:57.770770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:57.770770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:01:57.770770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:01:57.770770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:01:57.770770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:01:57.770770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:01:57.770770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:01:57.770770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 12:01:57.770770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:57.770770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 439/879] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Aug 12 12:01:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:57.776104 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f24bec-f7a4-4b8b-8cd7-6c80b4007197 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:57.819553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:57.881147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f24bec-f7a4-4b8b-8cd7-6c80b4007197 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:57.916139 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307748f9-3c8e-4af1-99ed-791e8dfb1dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:57.916827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307748f9-3c8e-4af1-99ed-791e8dfb1dbe None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:57.917106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307748f9-3c8e-4af1-99ed-791e8dfb1dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:57.917374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307748f9-3c8e-4af1-99ed-791e8dfb1dbe None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:57.918027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307748f9-3c8e-4af1-99ed-791e8dfb1dbe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:57.918323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307748f9-3c8e-4af1-99ed-791e8dfb1dbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:57.919683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307748f9-3c8e-4af1-99ed-791e8dfb1dbe None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:57.929046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 441/880] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Aug 12 12:01:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:57.934554 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb4a95-98cb-4dd0-937b-d9f8a9902f0b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:58.036272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb4a95-98cb-4dd0-937b-d9f8a9902f0b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:58.065394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:58.070247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872965ff-4f0f-47c4-9ff8-e9da6686fc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:58.070911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872965ff-4f0f-47c4-9ff8-e9da6686fc5b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:58.071192 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872965ff-4f0f-47c4-9ff8-e9da6686fc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:58.071473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872965ff-4f0f-47c4-9ff8-e9da6686fc5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:58.082927 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872965ff-4f0f-47c4-9ff8-e9da6686fc5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:58.084378 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872965ff-4f0f-47c4-9ff8-e9da6686fc5b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-872965ff-4f0f-47c4-9ff8-e9da6686fc5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:58.098644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:58.099594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:58.100913 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:58.100913 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:58.100913 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:58.100913 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 440/881] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:58.105735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307748f9-3c8e-4af1-99ed-791e8dfb1dbe None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:58.201006 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:58.210978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307748f9-3c8e-4af1-99ed-791e8dfb1dbe None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:58.245561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a283d4-dc11-4bca-ba01-fdbd2387c015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:58.246169 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a283d4-dc11-4bca-ba01-fdbd2387c015 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:58.246446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a283d4-dc11-4bca-ba01-fdbd2387c015 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:58.246717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a283d4-dc11-4bca-ba01-fdbd2387c015 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:58.247365 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a283d4-dc11-4bca-ba01-fdbd2387c015 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:58.247664 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a283d4-dc11-4bca-ba01-fdbd2387c015 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:58.249066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a283d4-dc11-4bca-ba01-fdbd2387c015 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:58.261978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 442/882] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:58.267983 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872965ff-4f0f-47c4-9ff8-e9da6686fc5b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:58.373205 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872965ff-4f0f-47c4-9ff8-e9da6686fc5b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:58.410814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e5c4f-3713-4abe-b0d7-6a912e4b5d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:58.411498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e5c4f-3713-4abe-b0d7-6a912e4b5d36 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:58.411778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e5c4f-3713-4abe-b0d7-6a912e4b5d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:58.412048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e5c4f-3713-4abe-b0d7-6a912e4b5d36 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:58.420559 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e5c4f-3713-4abe-b0d7-6a912e4b5d36 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:58.422079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e5c4f-3713-4abe-b0d7-6a912e4b5d36 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-448e5c4f-3713-4abe-b0d7-6a912e4b5d36 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:01:58.434013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:58.436364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:58.439269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:58.439269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:58.439269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:01:58.439269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:01:58.439269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:01:58.439269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:01:58.439269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:01:58.439269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:01:58.439269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 12:01:58.439269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:58.439269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 441/883] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 12 12:01:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:58.446058 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a283d4-dc11-4bca-ba01-fdbd2387c015 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:58.547049 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a283d4-dc11-4bca-ba01-fdbd2387c015 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:58.583424 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0d543-cb35-489d-b236-b59d0f296dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:58.584255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0d543-cb35-489d-b236-b59d0f296dee None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:58.584705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0d543-cb35-489d-b236-b59d0f296dee None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:58.585117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0d543-cb35-489d-b236-b59d0f296dee None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:58.585998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0d543-cb35-489d-b236-b59d0f296dee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:58.586442 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0d543-cb35-489d-b236-b59d0f296dee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:58.587992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0d543-cb35-489d-b236-b59d0f296dee None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:58.605787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 443/884] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Aug 12 12:01:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:58.612333 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e5c4f-3713-4abe-b0d7-6a912e4b5d36 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:58.665039 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:58.720319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e5c4f-3713-4abe-b0d7-6a912e4b5d36 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:58.735949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7e492-484f-4e22-8020-49aaf5191608 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:58.736507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7e492-484f-4e22-8020-49aaf5191608 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:58.737024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7e492-484f-4e22-8020-49aaf5191608 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:58.739533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 444/885] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:01:58.745014 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fb17b-feb6-434b-bbad-d1fbc566b089 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:58.745992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fb17b-feb6-434b-bbad-d1fbc566b089 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:58.746468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fb17b-feb6-434b-bbad-d1fbc566b089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:58.762508 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b3c9c-634f-4d75-9de4-a9edae35b0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:58.763117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b3c9c-634f-4d75-9de4-a9edae35b0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:58.763394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b3c9c-634f-4d75-9de4-a9edae35b0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:58.763664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b3c9c-634f-4d75-9de4-a9edae35b0a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:58.769948 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-cc7fb17b-feb6-434b-bbad-d1fbc566b089 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:58.771390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-cc7fb17b-feb6-434b-bbad-d1fbc566b089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:58.774177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:58.844564 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b3c9c-634f-4d75-9de4-a9edae35b0a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:58.846370 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b3c9c-634f-4d75-9de4-a9edae35b0a9 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:58.864664 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 445/886] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:01:58.898660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:58.959442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 442/887] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:01:58 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 12 12:01:58.964757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fb17b-feb6-434b-bbad-d1fbc566b089 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:59.067946 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fb17b-feb6-434b-bbad-d1fbc566b089 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:59.103553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bdaa3-8986-47bf-86ad-1eed79d9f678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:59.104185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bdaa3-8986-47bf-86ad-1eed79d9f678 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:59.104463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bdaa3-8986-47bf-86ad-1eed79d9f678 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:59.104765 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bdaa3-8986-47bf-86ad-1eed79d9f678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:59.115655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58bdaa3-8986-47bf-86ad-1eed79d9f678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:59.117181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58bdaa3-8986-47bf-86ad-1eed79d9f678 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:59.117820 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-e58bdaa3-8986-47bf-86ad-1eed79d9f678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:01:59.132609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:59.133619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:01:59.134489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:01:59.134489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:01:59.134489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:59.134489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 446/888] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:59.139661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b3c9c-634f-4d75-9de4-a9edae35b0a9 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:59.161106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bdaa3-8986-47bf-86ad-1eed79d9f678 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:59.270427 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b3c9c-634f-4d75-9de4-a9edae35b0a9 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:59.289023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bdaa3-8986-47bf-86ad-1eed79d9f678 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:59.305089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919dcc0-6c4a-4831-8752-ca2d167e8e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:59.305696 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919dcc0-6c4a-4831-8752-ca2d167e8e4c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:59.305976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919dcc0-6c4a-4831-8752-ca2d167e8e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:59.306255 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919dcc0-6c4a-4831-8752-ca2d167e8e4c None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:59.306895 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919dcc0-6c4a-4831-8752-ca2d167e8e4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:59.307199 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919dcc0-6c4a-4831-8752-ca2d167e8e4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:59.308556 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919dcc0-6c4a-4831-8752-ca2d167e8e4c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:59.320528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 443/889] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:59.321661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c714d-e907-4021-9f94-6c277ae617bb None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:59.321944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c714d-e907-4021-9f94-6c277ae617bb None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:59.322217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c714d-e907-4021-9f94-6c277ae617bb None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:59.323402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 444/890] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:59.324401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1cb7f-7cb5-40e7-b25c-e38e0e6f2d02 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:59.324819 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1cb7f-7cb5-40e7-b25c-e38e0e6f2d02 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:59.325004 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1cb7f-7cb5-40e7-b25c-e38e0e6f2d02 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:59.326262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 445/891] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:01:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:01:59.328770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1cb7f-7cb5-40e7-b25c-e38e0e6f2d02 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:59.330507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02c221-aea1-4a89-9a7f-3f70a9943b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:59.330933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:59.332032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02c221-aea1-4a89-9a7f-3f70a9943b5c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:59.332319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02c221-aea1-4a89-9a7f-3f70a9943b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:59.332598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02c221-aea1-4a89-9a7f-3f70a9943b5c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:59.333228 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02c221-aea1-4a89-9a7f-3f70a9943b5c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:59.334689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02c221-aea1-4a89-9a7f-3f70a9943b5c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-0e02c221-aea1-4a89-9a7f-3f70a9943b5c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:59.343659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:59.344926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:59.346063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:59.346063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:59.346063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:59.346063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:59.346063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:59.346063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:59.346063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:59.346063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:59.346063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:59.346063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:59.346063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:01:59.346063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:59.346063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 12:01:59.346063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:59.346063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 447/892] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:01:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:01:59.346840 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed9a76-14e9-4335-a120-f3b33a249b5e None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:59.346840 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed9a76-14e9-4335-a120-f3b33a249b5e None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:59.346840 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed9a76-14e9-4335-a120-f3b33a249b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:59.364987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-feed9a76-14e9-4335-a120-f3b33a249b5e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:59.366195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-feed9a76-14e9-4335-a120-f3b33a249b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:59.414699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:59.430135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1cb7f-7cb5-40e7-b25c-e38e0e6f2d02 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:59.431615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 448/893] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:01:59.432855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b968e-483c-4009-b584-ba5009c927f0 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:59.433138 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b968e-483c-4009-b584-ba5009c927f0 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:59.433412 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b968e-483c-4009-b584-ba5009c927f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:59.450996 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-3f9b968e-483c-4009-b584-ba5009c927f0 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:01:59.452241 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-3f9b968e-483c-4009-b584-ba5009c927f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:01:59.466621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475205ac-cf43-4821-b2b5-8956b4f78e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:59.467337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475205ac-cf43-4821-b2b5-8956b4f78e5d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:59.467513 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475205ac-cf43-4821-b2b5-8956b4f78e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:59.467782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475205ac-cf43-4821-b2b5-8956b4f78e5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:59.475377 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475205ac-cf43-4821-b2b5-8956b4f78e5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:59.476832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475205ac-cf43-4821-b2b5-8956b4f78e5d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:59.487814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-475205ac-cf43-4821-b2b5-8956b4f78e5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:59.487814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:59.487814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:59.487814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:59.487814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:59.487814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:59.487814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:59.487814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:59.487814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:59.487814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:59.487814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:59.487814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:59.487814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:01:59.487814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:01:59.487814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:59.488693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:59.489573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:01:59.489573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:01:59.489573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:01:59.489573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:01:59.489573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:01:59.489573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:01:59.489573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:01:59.489573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:01:59.489573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 446/894] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:01:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:59.490043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475205ac-cf43-4821-b2b5-8956b4f78e5d None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:59.532687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 449/895] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:01:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:01:59.532687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b968e-483c-4009-b584-ba5009c927f0 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:59.561709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:59.610447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475205ac-cf43-4821-b2b5-8956b4f78e5d None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:59.654133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275b871-a8ac-46ec-aaac-fd6e5ad58acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:59.655137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275b871-a8ac-46ec-aaac-fd6e5ad58acf None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:59.655424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275b871-a8ac-46ec-aaac-fd6e5ad58acf None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:59.655698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275b871-a8ac-46ec-aaac-fd6e5ad58acf None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:59.657133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275b871-a8ac-46ec-aaac-fd6e5ad58acf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:59.657472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275b871-a8ac-46ec-aaac-fd6e5ad58acf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:59.659573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275b871-a8ac-46ec-aaac-fd6e5ad58acf None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:59.669962 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 447/896] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:01:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:59.670112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b968e-483c-4009-b584-ba5009c927f0 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:59.671574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275b871-a8ac-46ec-aaac-fd6e5ad58acf None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:59.684773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:59.708132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7241384-4c25-4f43-a638-e429afbeff4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:59.708767 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7241384-4c25-4f43-a638-e429afbeff4a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:59.709049 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7241384-4c25-4f43-a638-e429afbeff4a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:59.709322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7241384-4c25-4f43-a638-e429afbeff4a None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:59.710288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7241384-4c25-4f43-a638-e429afbeff4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:59.710397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7241384-4c25-4f43-a638-e429afbeff4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:59.711913 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7241384-4c25-4f43-a638-e429afbeff4a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:59.725382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 450/897] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:01:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:59.726437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7241384-4c25-4f43-a638-e429afbeff4a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:59.788797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275b871-a8ac-46ec-aaac-fd6e5ad58acf None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:59.816562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:59.824491 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4262a6-7318-4c2e-a252-7a0d7a7b8372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:59.825133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4262a6-7318-4c2e-a252-7a0d7a7b8372 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:59.825414 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4262a6-7318-4c2e-a252-7a0d7a7b8372 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:59.825685 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4262a6-7318-4c2e-a252-7a0d7a7b8372 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:59.826310 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4262a6-7318-4c2e-a252-7a0d7a7b8372 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:59.826612 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4262a6-7318-4c2e-a252-7a0d7a7b8372 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:59.827967 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4262a6-7318-4c2e-a252-7a0d7a7b8372 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:59.832582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 448/898] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:01:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:59.833019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7241384-4c25-4f43-a638-e429afbeff4a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:59.833390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4262a6-7318-4c2e-a252-7a0d7a7b8372 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:59.869981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8097fa-f197-426d-925b-ba9d3c272017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:59.870541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8097fa-f197-426d-925b-ba9d3c272017 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:59.870826 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8097fa-f197-426d-925b-ba9d3c272017 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:59.871136 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8097fa-f197-426d-925b-ba9d3c272017 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:59.871725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8097fa-f197-426d-925b-ba9d3c272017 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:59.873139 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8097fa-f197-426d-925b-ba9d3c272017 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-4d8097fa-f197-426d-925b-ba9d3c272017 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:01:59.881892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:01:59.882931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:01:59.883762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:01:59.883762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:01:59.883762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:01:59.883762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:01:59.883762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:01:59.883762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:01:59.883762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:01:59.883762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:01:59.883762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:01:59.883762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:01:59.883762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:01:59.883762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:01:59.883762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 12:01:59.883762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:01:59.883762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 451/899] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 12 12:01:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:01:59.884543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8097fa-f197-426d-925b-ba9d3c272017 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:59.926545 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:01:59.932624 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4262a6-7318-4c2e-a252-7a0d7a7b8372 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:01:59.966299 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c745b-b6a2-442a-9539-7a67637dd81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:01:59.966903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c745b-b6a2-442a-9539-7a67637dd81c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:01:59.967188 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c745b-b6a2-442a-9539-7a67637dd81c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:01:59.967461 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c745b-b6a2-442a-9539-7a67637dd81c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:01:59.968189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c745b-b6a2-442a-9539-7a67637dd81c None admin] RBAC: Adding query filter params (role.id=ff702c6529bc4a57a4f3bfe2b4973444, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=ab4e815bbe7f4ff880f4cc633da1ed8c) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:01:59.968495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c745b-b6a2-442a-9539-7a67637dd81c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:01:59.970823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c745b-b6a2-442a-9539-7a67637dd81c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:01:59.976085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 449/900] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:01:59 2022] GET /identity/v3/role_assignments?user.id=ab4e815bbe7f4ff880f4cc633da1ed8c&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=ff702c6529bc4a57a4f3bfe2b4973444 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:01:59.976889 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c745b-b6a2-442a-9539-7a67637dd81c None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:01:59.986760 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8097fa-f197-426d-925b-ba9d3c272017 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:00.028000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e1438-1ce8-4ffd-9960-f430e795f49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:00.028719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e1438-1ce8-4ffd-9960-f430e795f49f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:00.029009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e1438-1ce8-4ffd-9960-f430e795f49f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:00.029280 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e1438-1ce8-4ffd-9960-f430e795f49f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:00.029889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e1438-1ce8-4ffd-9960-f430e795f49f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:00.031518 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e1438-1ce8-4ffd-9960-f430e795f49f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-f37e1438-1ce8-4ffd-9960-f430e795f49f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:00.041701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:00.048191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:00.049451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:00.049451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:00.049451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:00.049451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:00.049451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:00.049451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:00.049451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:00.049451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:00.049451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:00.049451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:00.049451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:02:00.049451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:00.049451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 12:02:00.049451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:00.049451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 452/901] 10.222.0.27 () {56 vars in 1132 bytes} [Fri Aug 12 12:01:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:00.050260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e1438-1ce8-4ffd-9960-f430e795f49f None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:00.054726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:00.103258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c745b-b6a2-442a-9539-7a67637dd81c None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:00.143187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ade57-98ab-4deb-ba0e-92417546ca0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:00.143721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ade57-98ab-4deb-ba0e-92417546ca0e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:00.144007 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ade57-98ab-4deb-ba0e-92417546ca0e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:00.144282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ade57-98ab-4deb-ba0e-92417546ca0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:00.145379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ade57-98ab-4deb-ba0e-92417546ca0e None admin] RBAC: Adding query filter params (interface=public, service_id=0321b2ca12ce4a269b467d5252046e8d, region_id=RegionOne) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:00.145487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ade57-98ab-4deb-ba0e-92417546ca0e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:00.146790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ade57-98ab-4deb-ba0e-92417546ca0e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:00.156498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 450/902] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Aug 12 12:01:59 2022] GET /identity/v3/endpoints?service_id=0321b2ca12ce4a269b467d5252046e8d&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:00.156498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ade57-98ab-4deb-ba0e-92417546ca0e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:00.163368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e1438-1ce8-4ffd-9960-f430e795f49f None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:00.165152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:00.199987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8330702-e436-4713-9749-5e220d895d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:00.200593 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8330702-e436-4713-9749-5e220d895d05 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:00.200905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8330702-e436-4713-9749-5e220d895d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:00.201178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8330702-e436-4713-9749-5e220d895d05 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:00.201855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8330702-e436-4713-9749-5e220d895d05 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:00.202202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8330702-e436-4713-9749-5e220d895d05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:00.204085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8330702-e436-4713-9749-5e220d895d05 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:00.208876 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 453/903] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 12 12:02:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:00.209674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8330702-e436-4713-9749-5e220d895d05 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:00.277485 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ade57-98ab-4deb-ba0e-92417546ca0e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:00.299961 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:00.310498 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8330702-e436-4713-9749-5e220d895d05 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:00.315143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef1d77-c018-4e1b-a174-e556f9ed00b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:00.315744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef1d77-c018-4e1b-a174-e556f9ed00b9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:00.316020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef1d77-c018-4e1b-a174-e556f9ed00b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:00.316287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef1d77-c018-4e1b-a174-e556f9ed00b9 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:00.317070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef1d77-c018-4e1b-a174-e556f9ed00b9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:00.317371 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef1d77-c018-4e1b-a174-e556f9ed00b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:00.321886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef1d77-c018-4e1b-a174-e556f9ed00b9 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:00.329655 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 451/904] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 12 12:02:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:00.330500 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef1d77-c018-4e1b-a174-e556f9ed00b9 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:00.346182 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c583d5-1c22-4e48-86b2-4087ff3bb515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:00.346785 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c583d5-1c22-4e48-86b2-4087ff3bb515 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:00.347063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c583d5-1c22-4e48-86b2-4087ff3bb515 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:00.347332 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c583d5-1c22-4e48-86b2-4087ff3bb515 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:00.347936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c583d5-1c22-4e48-86b2-4087ff3bb515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:00.349339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c583d5-1c22-4e48-86b2-4087ff3bb515 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:00.355773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 454/905] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 12 12:02:00 2022] GET /identity/v3/services => generated 1268 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:02:00.356541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c583d5-1c22-4e48-86b2-4087ff3bb515 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:00.393936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:00.468099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef1d77-c018-4e1b-a174-e556f9ed00b9 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:00.483808 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c583d5-1c22-4e48-86b2-4087ff3bb515 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:00.509534 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0494481-9574-4125-9133-1872b258a12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:00.510141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0494481-9574-4125-9133-1872b258a12b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:00.510424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0494481-9574-4125-9133-1872b258a12b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:00.510701 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0494481-9574-4125-9133-1872b258a12b None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:00.511367 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0494481-9574-4125-9133-1872b258a12b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:00.511674 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0494481-9574-4125-9133-1872b258a12b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:00.513078 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0494481-9574-4125-9133-1872b258a12b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:00.518018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 452/906] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 12 12:02:00 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:00.524165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cec746-3092-46e5-b5b9-852c246e647e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:00.524824 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cec746-3092-46e5-b5b9-852c246e647e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:00.525111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cec746-3092-46e5-b5b9-852c246e647e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:00.525385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cec746-3092-46e5-b5b9-852c246e647e None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:00.526387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cec746-3092-46e5-b5b9-852c246e647e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:00.526387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cec746-3092-46e5-b5b9-852c246e647e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:00.527760 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cec746-3092-46e5-b5b9-852c246e647e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:00.539235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 455/907] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 12 12:02:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:00.540506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0494481-9574-4125-9133-1872b258a12b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:00.598320 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:00.657306 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0494481-9574-4125-9133-1872b258a12b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:00.693351 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1cd34-a54c-4dd9-80de-eb094b5bd75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:00.694008 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1cd34-a54c-4dd9-80de-eb094b5bd75d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:00.694287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1cd34-a54c-4dd9-80de-eb094b5bd75d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:00.694558 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1cd34-a54c-4dd9-80de-eb094b5bd75d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:00.695314 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1cd34-a54c-4dd9-80de-eb094b5bd75d None admin] RBAC: Adding query filter params (interface=public, service_id=739158e32111435db44487bea455da64, region_id=RegionOne) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:00.695623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1cd34-a54c-4dd9-80de-eb094b5bd75d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:00.697012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1cd34-a54c-4dd9-80de-eb094b5bd75d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:00.702621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 453/908] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Aug 12 12:02:00 2022] GET /identity/v3/endpoints?service_id=739158e32111435db44487bea455da64&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:00.707994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cec746-3092-46e5-b5b9-852c246e647e None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:00.815185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:00.820798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cec746-3092-46e5-b5b9-852c246e647e None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:00.855432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6197a0-0161-4ee3-9c62-924edb1fa1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:00.856033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6197a0-0161-4ee3-9c62-924edb1fa1bb None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:00.856310 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6197a0-0161-4ee3-9c62-924edb1fa1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:00.856582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6197a0-0161-4ee3-9c62-924edb1fa1bb None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:00.857221 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6197a0-0161-4ee3-9c62-924edb1fa1bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:00.858703 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6197a0-0161-4ee3-9c62-924edb1fa1bb None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:00.865797 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 456/909] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 12 12:02:00 2022] GET /identity/v3/services => generated 1268 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:02:01.923140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5791238-2d67-4e7c-bfe1-7c4e9843b611 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:01.924190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5791238-2d67-4e7c-bfe1-7c4e9843b611 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:01.924744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5791238-2d67-4e7c-bfe1-7c4e9843b611 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:01.926565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 454/910] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:01.932237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e0a99-5a30-4af9-9fb5-f38926eb8b2b None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:01.933105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e0a99-5a30-4af9-9fb5-f38926eb8b2b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:01.933608 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e0a99-5a30-4af9-9fb5-f38926eb8b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:01.952432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9a8e0a99-5a30-4af9-9fb5-f38926eb8b2b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:01.954205 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9a8e0a99-5a30-4af9-9fb5-f38926eb8b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:02.025933 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 457/911] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:01 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:02.303976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5791238-2d67-4e7c-bfe1-7c4e9843b611 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:02.417010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5791238-2d67-4e7c-bfe1-7c4e9843b611 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:02.431275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:02.433005 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50f3f-0c5f-4fba-8ba0-f19eeeab4bc0 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:02.433294 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50f3f-0c5f-4fba-8ba0-f19eeeab4bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:02.433576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50f3f-0c5f-4fba-8ba0-f19eeeab4bc0 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:02.435171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 458/912] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:02.442388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217cf20-27d2-40cc-882c-4e8ae0bd173d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:02.442961 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217cf20-27d2-40cc-882c-4e8ae0bd173d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:02.443451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217cf20-27d2-40cc-882c-4e8ae0bd173d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:02.458883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd6484-3cbe-4788-b27a-50e65b2c497e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:02.459518 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd6484-3cbe-4788-b27a-50e65b2c497e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:02.459831 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd6484-3cbe-4788-b27a-50e65b2c497e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:02.460164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd6484-3cbe-4788-b27a-50e65b2c497e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:02.464849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f217cf20-27d2-40cc-882c-4e8ae0bd173d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:02.466807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f217cf20-27d2-40cc-882c-4e8ae0bd173d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:02.472464 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd6484-3cbe-4788-b27a-50e65b2c497e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-2acd6484-3cbe-4788-b27a-50e65b2c497e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:02:02.477272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:02.478213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:02.479032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:02.479032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:02.479032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:02.479032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:02.479032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:02:02.479032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:02:02.479032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:02:02.479032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:02:02.479032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:02:02.479032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:02:02.479032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 12:02:02.479032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:02.479032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 455/913] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Aug 12 12:02:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:02.484700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd6484-3cbe-4788-b27a-50e65b2c497e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:02.544602 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 459/914] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:02:02.548840 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205d776-c368-4925-82e8-dcd720889e9d None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:02.548840 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205d776-c368-4925-82e8-dcd720889e9d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:02.548840 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205d776-c368-4925-82e8-dcd720889e9d None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:02.548840 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 460/915] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:02.554873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37b30f-6fb8-480d-b952-6d225008e860 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:02.555094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37b30f-6fb8-480d-b952-6d225008e860 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:02.555362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37b30f-6fb8-480d-b952-6d225008e860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:02.574142 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-7e37b30f-6fb8-480d-b952-6d225008e860 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:02.575336 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7e37b30f-6fb8-480d-b952-6d225008e860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:02.589614 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:02.598574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd6484-3cbe-4788-b27a-50e65b2c497e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:02.634766 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf5832-e1fb-4312-a1fe-f83a15bcf572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:02.635658 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf5832-e1fb-4312-a1fe-f83a15bcf572 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:02.636150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf5832-e1fb-4312-a1fe-f83a15bcf572 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:02.636603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf5832-e1fb-4312-a1fe-f83a15bcf572 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:02.637535 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf5832-e1fb-4312-a1fe-f83a15bcf572 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:02.638020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf5832-e1fb-4312-a1fe-f83a15bcf572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:02.639554 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf5832-e1fb-4312-a1fe-f83a15bcf572 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:02.653227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 461/916] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:02:02.653504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 456/917] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Aug 12 12:02:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:02.659782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37b30f-6fb8-480d-b952-6d225008e860 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:02.764211 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37b30f-6fb8-480d-b952-6d225008e860 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:02.797000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f81e4-4ad1-4e14-91a1-4cc098c6ee04 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:02.797360 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f81e4-4ad1-4e14-91a1-4cc098c6ee04 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:02.797668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f81e4-4ad1-4e14-91a1-4cc098c6ee04 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:02.800253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3bcb-f999-4c58-b5b8-f64cf12ecb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:02.800981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3bcb-f999-4c58-b5b8-f64cf12ecb3d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:02.801311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3bcb-f999-4c58-b5b8-f64cf12ecb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:02.801588 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3bcb-f999-4c58-b5b8-f64cf12ecb3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:02.813367 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 457/918] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:02 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:02:02.814188 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c3bcb-f999-4c58-b5b8-f64cf12ecb3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:02.815668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c3bcb-f999-4c58-b5b8-f64cf12ecb3d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:02.816386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24fe1e-1240-49bf-b428-e32eec1b906c None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:02.816386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24fe1e-1240-49bf-b428-e32eec1b906c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:02.816561 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24fe1e-1240-49bf-b428-e32eec1b906c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:02.825837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-f19c3bcb-f999-4c58-b5b8-f64cf12ecb3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:02:02.833450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:02.840675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:02:02.841607 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:02:02.841607 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:02.841607 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:02.841789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-bc24fe1e-1240-49bf-b428-e32eec1b906c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:02.841863 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 462/919] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:02.841863 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3bcb-f999-4c58-b5b8-f64cf12ecb3d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:02.841984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-bc24fe1e-1240-49bf-b428-e32eec1b906c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:02.869832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:02.914173 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 458/920] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:02.921689 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24fe1e-1240-49bf-b428-e32eec1b906c None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:02.949157 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3bcb-f999-4c58-b5b8-f64cf12ecb3d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:02.986088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948b85f-f9c7-4718-b910-631348423fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:02.986953 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948b85f-f9c7-4718-b910-631348423fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:02.987362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948b85f-f9c7-4718-b910-631348423fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:02.987753 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948b85f-f9c7-4718-b910-631348423fc3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:02.988414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948b85f-f9c7-4718-b910-631348423fc3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:02.989829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948b85f-f9c7-4718-b910-631348423fc3 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-7948b85f-f9c7-4718-b910-631348423fc3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:02.999023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:03.000051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:03.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:03.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:03.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:03.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:03.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:03.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:03.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:03.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:03.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:03.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:03.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:02:03.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:03.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 12 12:02:03.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:03.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 463/921] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:02:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:03.002735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948b85f-f9c7-4718-b910-631348423fc3 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:03.043466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24fe1e-1240-49bf-b428-e32eec1b906c None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:03.069878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:03.080723 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace741ae-90f9-47e4-8a15-b89471817183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:03.081461 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace741ae-90f9-47e4-8a15-b89471817183 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:03.081758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace741ae-90f9-47e4-8a15-b89471817183 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:03.087480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace741ae-90f9-47e4-8a15-b89471817183 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:03.087480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace741ae-90f9-47e4-8a15-b89471817183 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:03.087480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace741ae-90f9-47e4-8a15-b89471817183 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:03.087480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace741ae-90f9-47e4-8a15-b89471817183 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:03.100418 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 459/922] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:03.100418 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace741ae-90f9-47e4-8a15-b89471817183 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:03.111981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948b85f-f9c7-4718-b910-631348423fc3 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:03.126743 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:03.148348 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e976c-2d8e-4216-a6dc-4f442739aa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:03.148967 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e976c-2d8e-4216-a6dc-4f442739aa3f None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:03.149249 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e976c-2d8e-4216-a6dc-4f442739aa3f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:03.149525 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3e976c-2d8e-4216-a6dc-4f442739aa3f None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:03.150169 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e976c-2d8e-4216-a6dc-4f442739aa3f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:03.151530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3e976c-2d8e-4216-a6dc-4f442739aa3f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:03.200801 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace741ae-90f9-47e4-8a15-b89471817183 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:03.228120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 464/923] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 12 12:02:03 2022] POST /identity/v3/services => generated 241 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:02:03.229350 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e976c-2d8e-4216-a6dc-4f442739aa3f None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:03.245931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f78d5-ebd5-49cb-a103-8ea12cbed9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:03.246827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f78d5-ebd5-49cb-a103-8ea12cbed9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:03.247109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f78d5-ebd5-49cb-a103-8ea12cbed9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:03.247380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f78d5-ebd5-49cb-a103-8ea12cbed9b6 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:03.248214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f78d5-ebd5-49cb-a103-8ea12cbed9b6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:03.248325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f78d5-ebd5-49cb-a103-8ea12cbed9b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:03.249786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f78d5-ebd5-49cb-a103-8ea12cbed9b6 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:03.257657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 460/924] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:02:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:03.258844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f78d5-ebd5-49cb-a103-8ea12cbed9b6 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:03.351115 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3e976c-2d8e-4216-a6dc-4f442739aa3f None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:03.357217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:03.375630 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f78d5-ebd5-49cb-a103-8ea12cbed9b6 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:03.380508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:03.392748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec9ea7-d65d-4a74-b20d-64cdcc9a171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:03.393378 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec9ea7-d65d-4a74-b20d-64cdcc9a171f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:03.393653 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec9ea7-d65d-4a74-b20d-64cdcc9a171f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:03.393920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec9ea7-d65d-4a74-b20d-64cdcc9a171f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:03.401421 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec9ea7-d65d-4a74-b20d-64cdcc9a171f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:03.402919 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec9ea7-d65d-4a74-b20d-64cdcc9a171f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:03.413948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95ef7b-e441-4cdf-81a3-7841dd5a9fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:03.414571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95ef7b-e441-4cdf-81a3-7841dd5a9fff None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:03.414850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95ef7b-e441-4cdf-81a3-7841dd5a9fff None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:03.415120 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95ef7b-e441-4cdf-81a3-7841dd5a9fff None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:03.415706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95ef7b-e441-4cdf-81a3-7841dd5a9fff None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-7aec9ea7-d65d-4a74-b20d-64cdcc9a171f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:02:03.416851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:03.417716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:03.418632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:03.418632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:03.418632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:02:03.418632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:02:03.418632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:02:03.418632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:02:03.418632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:02:03.418632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:02:03.418632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 12:02:03.418632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:03.419155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95ef7b-e441-4cdf-81a3-7841dd5a9fff None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:03.419223 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 465/925] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 12 12:02:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:03.419223 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec9ea7-d65d-4a74-b20d-64cdcc9a171f None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-fc95ef7b-e441-4cdf-81a3-7841dd5a9fff None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:03.432498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:03.433775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:03.434633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:03.434633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:03.434633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:03.434633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:03.434633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:03.434633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:03.434633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:03.434633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:03.434633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:03.434633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:03.434633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:02:03.434633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:03.434633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 12 12:02:03.434633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:03.434633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 461/926] 10.222.0.27 () {56 vars in 1132 bytes} [Fri Aug 12 12:02:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:03.435528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95ef7b-e441-4cdf-81a3-7841dd5a9fff None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:03.533792 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec9ea7-d65d-4a74-b20d-64cdcc9a171f None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:03.545188 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95ef7b-e441-4cdf-81a3-7841dd5a9fff None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:03.570604 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a6ed1-6fd0-474b-8a27-c451160af6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:03.571270 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a6ed1-6fd0-474b-8a27-c451160af6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:03.571485 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a6ed1-6fd0-474b-8a27-c451160af6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:03.571755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a6ed1-6fd0-474b-8a27-c451160af6b7 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:03.572624 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a6ed1-6fd0-474b-8a27-c451160af6b7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:03.572748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a6ed1-6fd0-474b-8a27-c451160af6b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:03.574078 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a6ed1-6fd0-474b-8a27-c451160af6b7 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:03.583782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 466/927] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 12 12:02:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:03.583782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a6ed1-6fd0-474b-8a27-c451160af6b7 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:03.584993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b307520-d2f8-4bdd-a2e7-bf78d55006d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:03.585599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b307520-d2f8-4bdd-a2e7-bf78d55006d1 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:03.585875 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b307520-d2f8-4bdd-a2e7-bf78d55006d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:03.586203 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b307520-d2f8-4bdd-a2e7-bf78d55006d1 None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:03.586947 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b307520-d2f8-4bdd-a2e7-bf78d55006d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:03.587303 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b307520-d2f8-4bdd-a2e7-bf78d55006d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:03.588835 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b307520-d2f8-4bdd-a2e7-bf78d55006d1 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:03.612871 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 462/928] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Aug 12 12:02:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:03.612871 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b307520-d2f8-4bdd-a2e7-bf78d55006d1 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:03.619959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:03.640022 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:03.695324 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a6ed1-6fd0-474b-8a27-c451160af6b7 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:03.723215 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b307520-d2f8-4bdd-a2e7-bf78d55006d1 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:03.732471 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3f568-d23e-454c-96e7-774a6f86b9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:03.733147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3f568-d23e-454c-96e7-774a6f86b9fa None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:03.733433 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3f568-d23e-454c-96e7-774a6f86b9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:03.733704 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3f568-d23e-454c-96e7-774a6f86b9fa None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:03.734837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3f568-d23e-454c-96e7-774a6f86b9fa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:03.735209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3f568-d23e-454c-96e7-774a6f86b9fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:03.736938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3f568-d23e-454c-96e7-774a6f86b9fa None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:03.747250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 467/929] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 12 12:02:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:03.747250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3f568-d23e-454c-96e7-774a6f86b9fa None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:03.760449 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64109cc4-b163-4849-b3a0-5b1ceaf1ef98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:03.761107 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64109cc4-b163-4849-b3a0-5b1ceaf1ef98 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:03.761394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64109cc4-b163-4849-b3a0-5b1ceaf1ef98 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:03.761666 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64109cc4-b163-4849-b3a0-5b1ceaf1ef98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:03.762332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64109cc4-b163-4849-b3a0-5b1ceaf1ef98 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:03.763696 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64109cc4-b163-4849-b3a0-5b1ceaf1ef98 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:03.852766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3f568-d23e-454c-96e7-774a6f86b9fa None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:03.862745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 463/930] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:02:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:02:03.864080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64109cc4-b163-4849-b3a0-5b1ceaf1ef98 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:03.873731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:03.896144 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56b4a6-abb0-4c24-a54c-787b55ddfe57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:03.897227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56b4a6-abb0-4c24-a54c-787b55ddfe57 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:03.900218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56b4a6-abb0-4c24-a54c-787b55ddfe57 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:03.900218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56b4a6-abb0-4c24-a54c-787b55ddfe57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:03.913415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56b4a6-abb0-4c24-a54c-787b55ddfe57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:03.915704 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56b4a6-abb0-4c24-a54c-787b55ddfe57 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-bc56b4a6-abb0-4c24-a54c-787b55ddfe57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:02:03.935867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:03.936926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:02:03.938187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:02:03.938187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:03.938187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:03.940808 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 468/931] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:03.944262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56b4a6-abb0-4c24-a54c-787b55ddfe57 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:03.961428 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:04.005887 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64109cc4-b163-4849-b3a0-5b1ceaf1ef98 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:04.044472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357f9d6-350d-4c64-8d21-654088538f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:04.045163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357f9d6-350d-4c64-8d21-654088538f13 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:04.045485 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357f9d6-350d-4c64-8d21-654088538f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:04.045850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357f9d6-350d-4c64-8d21-654088538f13 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:04.046665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357f9d6-350d-4c64-8d21-654088538f13 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:04.047647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357f9d6-350d-4c64-8d21-654088538f13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:04.049535 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357f9d6-350d-4c64-8d21-654088538f13 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:04.056939 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 464/932] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 12 12:02:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:04.062669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357f9d6-350d-4c64-8d21-654088538f13 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:04.063804 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56b4a6-abb0-4c24-a54c-787b55ddfe57 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:04.105425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ca3f0-2e9e-45a6-8307-fbff185408bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:04.106030 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ca3f0-2e9e-45a6-8307-fbff185408bc None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:04.106313 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ca3f0-2e9e-45a6-8307-fbff185408bc None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:04.106634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ca3f0-2e9e-45a6-8307-fbff185408bc None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:04.107285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ca3f0-2e9e-45a6-8307-fbff185408bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:04.107589 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ca3f0-2e9e-45a6-8307-fbff185408bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:04.110038 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ca3f0-2e9e-45a6-8307-fbff185408bc None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:04.123433 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 469/933] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:04.129048 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ca3f0-2e9e-45a6-8307-fbff185408bc None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:04.155061 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:04.176827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357f9d6-350d-4c64-8d21-654088538f13 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:04.219739 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:04.231366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909fc11-21a4-4aa7-849f-7eb0d1769f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:04.231976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909fc11-21a4-4aa7-849f-7eb0d1769f9b None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:04.232257 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909fc11-21a4-4aa7-849f-7eb0d1769f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:04.232529 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909fc11-21a4-4aa7-849f-7eb0d1769f9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:04.233232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909fc11-21a4-4aa7-849f-7eb0d1769f9b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:04.235029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909fc11-21a4-4aa7-849f-7eb0d1769f9b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:04.260252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ca3f0-2e9e-45a6-8307-fbff185408bc None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:04.297624 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b626a23-4c06-4837-adf9-708f0ca6b020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:04.298335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b626a23-4c06-4837-adf9-708f0ca6b020 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:04.298698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b626a23-4c06-4837-adf9-708f0ca6b020 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:04.299315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b626a23-4c06-4837-adf9-708f0ca6b020 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:04.306621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b626a23-4c06-4837-adf9-708f0ca6b020 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:04.308274 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b626a23-4c06-4837-adf9-708f0ca6b020 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:04.316135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 465/934] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:02:04 2022] POST /identity/v3/endpoints => generated 347 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:02:04.327881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-3b626a23-4c06-4837-adf9-708f0ca6b020 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:02:04.327881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:04.327881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:04.327881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:04.327881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:04.327881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:04.327881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:04.327881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:04.327881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:04.327881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:04.327881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:04.327881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:04.327881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:02:04.327881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:02:04.327881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:04.329484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:04.330425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:02:04.330425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:02:04.330425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:02:04.330425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:02:04.330425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:02:04.330425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:02:04.330425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:02:04.330425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:04.333391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 470/935] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:02:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:04.338976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909fc11-21a4-4aa7-849f-7eb0d1769f9b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:04.465776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909fc11-21a4-4aa7-849f-7eb0d1769f9b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:04.503312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048065a-1d46-4c11-a653-ab246801450b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:04.504123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048065a-1d46-4c11-a653-ab246801450b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:04.504521 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048065a-1d46-4c11-a653-ab246801450b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:04.504907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048065a-1d46-4c11-a653-ab246801450b None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:04.505707 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048065a-1d46-4c11-a653-ab246801450b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:04.506114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048065a-1d46-4c11-a653-ab246801450b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:04.507743 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048065a-1d46-4c11-a653-ab246801450b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:04.521619 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 466/936] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:02:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:04.530117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b626a23-4c06-4837-adf9-708f0ca6b020 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:04.617341 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:04.642338 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b626a23-4c06-4837-adf9-708f0ca6b020 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:04.679212 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1b29e-f851-4f12-bd4f-4157ebfda298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:04.679879 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1b29e-f851-4f12-bd4f-4157ebfda298 None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:04.680165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1b29e-f851-4f12-bd4f-4157ebfda298 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:04.680441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1b29e-f851-4f12-bd4f-4157ebfda298 None admin] PATH_INFO: `/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/ab4e815bbe7f4ff880f4cc633da1ed8c/roles/ff702c6529bc4a57a4f3bfe2b4973444` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:04.682662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1b29e-f851-4f12-bd4f-4157ebfda298 None admin] RBAC: Authorizing `identity:create_grant(project_id=20c5570631a34c4ab3868eb1cd96e345, user_id=ab4e815bbe7f4ff880f4cc633da1ed8c, role_id=ff702c6529bc4a57a4f3bfe2b4973444)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:04.684100 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1b29e-f851-4f12-bd4f-4157ebfda298 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:04.782801 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 471/937] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:02:04 2022] PUT /identity/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/ab4e815bbe7f4ff880f4cc633da1ed8c/roles/ff702c6529bc4a57a4f3bfe2b4973444 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:02:05.221901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915ef01-3dfa-4536-922c-110e27fc4621 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:05.222459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915ef01-3dfa-4536-922c-110e27fc4621 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:05.222459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915ef01-3dfa-4536-922c-110e27fc4621 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:05.224005 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 467/938] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:05.229197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c650a-b123-407c-861c-58e2a0842467 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:05.229647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c650a-b123-407c-861c-58e2a0842467 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:05.229743 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c650a-b123-407c-861c-58e2a0842467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:05.250436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-269c650a-b123-407c-861c-58e2a0842467 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:05.251886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-269c650a-b123-407c-861c-58e2a0842467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:05.322074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 472/939] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:05.606691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915ef01-3dfa-4536-922c-110e27fc4621 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:05.651477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:05.714096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915ef01-3dfa-4536-922c-110e27fc4621 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:05.751341 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3d1cd-3fab-43f3-aad5-cf46a3696888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:05.752250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3d1cd-3fab-43f3-aad5-cf46a3696888 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:05.752707 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3d1cd-3fab-43f3-aad5-cf46a3696888 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:05.753133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3d1cd-3fab-43f3-aad5-cf46a3696888 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:05.753863 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3d1cd-3fab-43f3-aad5-cf46a3696888 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:05.755371 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3d1cd-3fab-43f3-aad5-cf46a3696888 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-55d3d1cd-3fab-43f3-aad5-cf46a3696888 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:05.764263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:05.765581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:05.766900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:05.766900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:05.766900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:05.766900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:05.766900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:05.766900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:05.766900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:05.766900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:05.766900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:05.766900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:05.766900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:02:05.766900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:05.766900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 12:02:05.766900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:05.769361 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 468/940] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 12 12:02:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:05.780675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c650a-b123-407c-861c-58e2a0842467 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:05.853027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:05.883148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c650a-b123-407c-861c-58e2a0842467 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:05.918339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3d66f-23c1-46a3-a121-3fcfd30b9731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:05.918949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3d66f-23c1-46a3-a121-3fcfd30b9731 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:05.919230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3d66f-23c1-46a3-a121-3fcfd30b9731 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:05.919504 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3d66f-23c1-46a3-a121-3fcfd30b9731 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:05.920140 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3d66f-23c1-46a3-a121-3fcfd30b9731 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:05.920445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3d66f-23c1-46a3-a121-3fcfd30b9731 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:05.922005 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3d66f-23c1-46a3-a121-3fcfd30b9731 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:05.927462 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 473/941] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 12 12:02:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:05.933854 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3d1cd-3fab-43f3-aad5-cf46a3696888 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:05.972357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04264c-5804-4ff2-9e86-575982333c75 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:05.972678 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04264c-5804-4ff2-9e86-575982333c75 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:05.972952 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04264c-5804-4ff2-9e86-575982333c75 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:05.974530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 474/942] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:05.979521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdd455-2bb6-4ffa-815f-b3c7ccb039b1 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:05.979813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdd455-2bb6-4ffa-815f-b3c7ccb039b1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:05.980093 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdd455-2bb6-4ffa-815f-b3c7ccb039b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:05.999281 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-28cdd455-2bb6-4ffa-815f-b3c7ccb039b1 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:06.001452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-28cdd455-2bb6-4ffa-815f-b3c7ccb039b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:06.048210 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3d1cd-3fab-43f3-aad5-cf46a3696888 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:06.050024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:06.072362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 475/943] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:02:06.089368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c999ef9-b384-4bf5-8d23-161f821b6eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:06.090171 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c999ef9-b384-4bf5-8d23-161f821b6eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:06.090618 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c999ef9-b384-4bf5-8d23-161f821b6eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:06.091051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c999ef9-b384-4bf5-8d23-161f821b6eb0 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:06.091822 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c999ef9-b384-4bf5-8d23-161f821b6eb0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:06.092319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c999ef9-b384-4bf5-8d23-161f821b6eb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:06.094038 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c999ef9-b384-4bf5-8d23-161f821b6eb0 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:06.099542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 469/944] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 12 12:02:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:06.105730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdd455-2bb6-4ffa-815f-b3c7ccb039b1 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:06.169527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:06.213936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdd455-2bb6-4ffa-815f-b3c7ccb039b1 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:06.251578 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0baaa-dc3a-495b-a054-e9464ee60543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:06.252227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0baaa-dc3a-495b-a054-e9464ee60543 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:06.252463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0baaa-dc3a-495b-a054-e9464ee60543 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:06.252763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0baaa-dc3a-495b-a054-e9464ee60543 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:06.253475 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0baaa-dc3a-495b-a054-e9464ee60543 None admin] RBAC: Adding query filter params (interface=public, service_id=b6a8123264e74a1fbfd27ee231001170, region_id=RegionOne) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:06.253785 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0baaa-dc3a-495b-a054-e9464ee60543 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:06.255165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0baaa-dc3a-495b-a054-e9464ee60543 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:06.263174 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 476/945] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Aug 12 12:02:06 2022] GET /identity/v3/endpoints?service_id=b6a8123264e74a1fbfd27ee231001170&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:06.269001 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c999ef9-b384-4bf5-8d23-161f821b6eb0 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:06.353134 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0baaa-dc3a-495b-a054-e9464ee60543 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:06.377091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c999ef9-b384-4bf5-8d23-161f821b6eb0 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:06.421625 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcfca5-b78e-4afe-a86b-4a3ca873a58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:06.422726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcfca5-b78e-4afe-a86b-4a3ca873a58d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:06.423183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcfca5-b78e-4afe-a86b-4a3ca873a58d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:06.423340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcfca5-b78e-4afe-a86b-4a3ca873a58d None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:06.424065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcfca5-b78e-4afe-a86b-4a3ca873a58d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:06.425285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcfca5-b78e-4afe-a86b-4a3ca873a58d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:06.432032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 470/946] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 12 12:02:06 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:02:06.433542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39cbcf-b8b2-40cc-9146-7048e7c8d2f8 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:06.434071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39cbcf-b8b2-40cc-9146-7048e7c8d2f8 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:06.434565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39cbcf-b8b2-40cc-9146-7048e7c8d2f8 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:06.436355 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 471/947] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:06.442349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215d679-ad87-4408-a119-cb48f53d79c5 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:06.443009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215d679-ad87-4408-a119-cb48f53d79c5 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:06.443499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215d679-ad87-4408-a119-cb48f53d79c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:06.464179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-0215d679-ad87-4408-a119-cb48f53d79c5 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:06.465949 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0215d679-ad87-4408-a119-cb48f53d79c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:06.481231 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0baaa-dc3a-495b-a054-e9464ee60543 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:06.496909 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:06.518479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:06.528360 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70f2d4-65ce-41d9-a720-9c7a6ae52945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:06.529005 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70f2d4-65ce-41d9-a720-9c7a6ae52945 None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:06.529297 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70f2d4-65ce-41d9-a720-9c7a6ae52945 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:06.529577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70f2d4-65ce-41d9-a720-9c7a6ae52945 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:06.530229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70f2d4-65ce-41d9-a720-9c7a6ae52945 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:06.532089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70f2d4-65ce-41d9-a720-9c7a6ae52945 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:06.535802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: WARNING keystone.server.flask.common [None req-ef70f2d4-65ce-41d9-a720-9c7a6ae52945 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 12 12:02:06.557329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 472/948] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:02:06.603577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.sql.core [None req-ef70f2d4-65ce-41d9-a720-9c7a6ae52945 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 12 12:02:06.603577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 12 12:02:06.603577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [parameters: {'id': '7aa2df78a04347e899765b0f5f1e4236', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 12 12:02:06.603577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81794) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 12 12:02:06.610811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-ef70f2d4-65ce-41d9-a720-9c7a6ae52945 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7aa2df78a04347e899765b0f5f1e4236.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7aa2df78a04347e899765b0f5f1e4236. Aug 12 12:02:06.610811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:06.610811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 12 12:02:06.610811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 12 12:02:06.610811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 12 12:02:06.610811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 12 12:02:06.610811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 12 12:02:06.610811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application result = self._query(query) Aug 12 12:02:06.610811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 12 12:02:06.610811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application conn.query(q) Aug 12 12:02:06.610811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 12 12:02:06.610811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 12 12:02:06.610811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application result.read() Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 12 12:02:06.611618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application next(self.gen) Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application yield resource Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application next(self.gen) Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self.session.rollback() Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self.force_reraise() Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise self.value Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application session.commit() Aug 12 12:02:06.612898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._prepare_impl() Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self.session.flush() Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._flush(objects) Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application compat.raise_( Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application flush_context.execute() Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 12 12:02:06.614090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rec.execute(self) Aug 12 12:02:06.614958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 12 12:02:06.614958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 12 12:02:06.614958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 12 12:02:06.614958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 12 12:02:06.614958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 12 12:02:06.614958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 12 12:02:06.614958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 12 12:02:06.614958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 12 12:02:06.614958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 12 12:02:06.614958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 12 12:02:06.614958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 12 12:02:06.614958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 12 12:02:06.614958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 12 12:02:06.614958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 12 12:02:06.614958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application result = self._query(query) Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application conn.query(q) Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application result.read() Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 12 12:02:06.615736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 12 12:02:06.617215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 12 12:02:06.617215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 12 12:02:06.617215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 12 12:02:06.617215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 12 12:02:06.617215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 12 12:02:06.617215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 12 12:02:06.617215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 12 12:02:06.617215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 12 12:02:06.617215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [parameters: {'id': '7aa2df78a04347e899765b0f5f1e4236', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 12 12:02:06.617215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 12 12:02:06.617215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:06.617215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:06.617215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:06.617215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:06.617215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:06.618755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:06.619870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:06.619870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 12 12:02:06.619870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 12 12:02:06.619870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:06.619870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:06.619870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 12 12:02:06.619870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 12 12:02:06.619870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7aa2df78a04347e899765b0f5f1e4236. Aug 12 12:02:06.619870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:06.619870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 477/949] 10.222.0.27 () {60 vars in 1140 bytes} [Fri Aug 12 12:02:06 2022] POST /identity/v3/projects => generated 262 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:02:06.620542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215d679-ad87-4408-a119-cb48f53d79c5 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:06.733837 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215d679-ad87-4408-a119-cb48f53d79c5 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:06.772658 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c7d2d-c5bd-4ad8-af8f-24028f60df95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:06.773285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c7d2d-c5bd-4ad8-af8f-24028f60df95 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:06.773569 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c7d2d-c5bd-4ad8-af8f-24028f60df95 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:06.773843 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c7d2d-c5bd-4ad8-af8f-24028f60df95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:06.781017 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c7d2d-c5bd-4ad8-af8f-24028f60df95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:06.783411 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c7d2d-c5bd-4ad8-af8f-24028f60df95 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:06.795918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-419c7d2d-c5bd-4ad8-af8f-24028f60df95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:02:06.795918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:06.795918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:06.795918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:06.795918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:06.795918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:06.795918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:06.795918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:06.795918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:06.795918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:06.795918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:06.795918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:06.795918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:02:06.795918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:02:06.795918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:06.804089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:06.813019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:02:06.813019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:02:06.813019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:02:06.813019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:02:06.813019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:02:06.813019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:02:06.813019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:02:06.813019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:06.813019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 473/950] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:02:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:06.814226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70f2d4-65ce-41d9-a720-9c7a6ae52945 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:06.843043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c7d2d-c5bd-4ad8-af8f-24028f60df95 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:06.869108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:06.926443 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70f2d4-65ce-41d9-a720-9c7a6ae52945 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:06.960540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c7d2d-c5bd-4ad8-af8f-24028f60df95 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:06.961508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b38b8e-6d81-4440-b987-f821bc21dd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:06.962190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b38b8e-6d81-4440-b987-f821bc21dd24 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:06.962516 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b38b8e-6d81-4440-b987-f821bc21dd24 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:06.962791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b38b8e-6d81-4440-b987-f821bc21dd24 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:06.963506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b38b8e-6d81-4440-b987-f821bc21dd24 None admin] RBAC: Adding query filter params (name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:06.963810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b38b8e-6d81-4440-b987-f821bc21dd24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:06.965479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b38b8e-6d81-4440-b987-f821bc21dd24 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:06.988157 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 478/951] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 12 12:02:06 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:06.989810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5307e4-83e8-4242-bb3e-03dc5864fd82 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:06.990307 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5307e4-83e8-4242-bb3e-03dc5864fd82 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:06.990804 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5307e4-83e8-4242-bb3e-03dc5864fd82 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:06.992279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 479/952] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:06.998145 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a3595-1778-4500-a86e-a27e82204219 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:06.998733 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a3595-1778-4500-a86e-a27e82204219 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:06.999182 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a3595-1778-4500-a86e-a27e82204219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:07.006781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ded56-323b-4c72-8e02-73af0d6f5934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:07.007522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ded56-323b-4c72-8e02-73af0d6f5934 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:07.007870 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ded56-323b-4c72-8e02-73af0d6f5934 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:07.008207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ded56-323b-4c72-8e02-73af0d6f5934 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:07.008912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ded56-323b-4c72-8e02-73af0d6f5934 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:07.010364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ded56-323b-4c72-8e02-73af0d6f5934 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-340ded56-323b-4c72-8e02-73af0d6f5934 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:07.021064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:07.022163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:07.025907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:07.025907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:07.025907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:07.025907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:07.025907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:07.025907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:07.025907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:07.025907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:07.025907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:07.025907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:07.025907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:02:07.025907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:07.025907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 12:02:07.025907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:07.025907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 474/953] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:02:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:02:07.028380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ded56-323b-4c72-8e02-73af0d6f5934 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:07.028900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-e33a3595-1778-4500-a86e-a27e82204219 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:07.030756 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e33a3595-1778-4500-a86e-a27e82204219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:07.079837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:07.117708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 480/954] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:02:07.150234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:07.168409 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ded56-323b-4c72-8e02-73af0d6f5934 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:07.204346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b6269-0835-4833-a4dc-bbc01367189e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:07.204975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b6269-0835-4833-a4dc-bbc01367189e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:07.205258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b6269-0835-4833-a4dc-bbc01367189e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:07.205529 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b6269-0835-4833-a4dc-bbc01367189e None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:07.206162 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b6269-0835-4833-a4dc-bbc01367189e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:07.206461 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b6269-0835-4833-a4dc-bbc01367189e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:07.207823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b6269-0835-4833-a4dc-bbc01367189e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:07.212739 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 475/955] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 12 12:02:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:07.219215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a3595-1778-4500-a86e-a27e82204219 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:07.325575 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a3595-1778-4500-a86e-a27e82204219 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:07.362513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6acf2-92bb-4ab0-bb0a-c28b4762395b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:07.363124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6acf2-92bb-4ab0-bb0a-c28b4762395b None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:07.363406 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6acf2-92bb-4ab0-bb0a-c28b4762395b None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:07.363681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6acf2-92bb-4ab0-bb0a-c28b4762395b None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:07.364318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6acf2-92bb-4ab0-bb0a-c28b4762395b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:07.364621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6acf2-92bb-4ab0-bb0a-c28b4762395b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:07.366024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6acf2-92bb-4ab0-bb0a-c28b4762395b None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:07.371175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 481/956] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 12 12:02:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:07.401136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b6269-0835-4833-a4dc-bbc01367189e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:07.524832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b6269-0835-4833-a4dc-bbc01367189e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:07.567449 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef8494-a0a7-4e37-b961-6118a0fd418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:07.568088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef8494-a0a7-4e37-b961-6118a0fd418a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:07.568372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef8494-a0a7-4e37-b961-6118a0fd418a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:07.568692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef8494-a0a7-4e37-b961-6118a0fd418a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:07.580533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef8494-a0a7-4e37-b961-6118a0fd418a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-3bef8494-a0a7-4e37-b961-6118a0fd418a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:02:07.585692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:07.586657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:07.587538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:07.587538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:07.587538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:07.587538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:07.587538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:02:07.587538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:02:07.587538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:02:07.587538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:02:07.587538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:02:07.587538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:02:07.587538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 12:02:07.587538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:07.587538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 476/957] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Aug 12 12:02:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:07.592925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6acf2-92bb-4ab0-bb0a-c28b4762395b None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:07.641245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:07.700390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6acf2-92bb-4ab0-bb0a-c28b4762395b None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:07.737637 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b0441-d422-47a3-abf5-a740a2dbd33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:07.738246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b0441-d422-47a3-abf5-a740a2dbd33a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:07.738532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b0441-d422-47a3-abf5-a740a2dbd33a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:07.738806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b0441-d422-47a3-abf5-a740a2dbd33a None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:07.739474 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b0441-d422-47a3-abf5-a740a2dbd33a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:07.739778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b0441-d422-47a3-abf5-a740a2dbd33a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:07.741185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b0441-d422-47a3-abf5-a740a2dbd33a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:07.749902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 482/958] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Aug 12 12:02:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:07.757808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef8494-a0a7-4e37-b961-6118a0fd418a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:07.770069 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:07.871937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef8494-a0a7-4e37-b961-6118a0fd418a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:07.906717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ab135-168e-47c9-8fdc-bd4383631d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:07.907710 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ab135-168e-47c9-8fdc-bd4383631d76 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:07.908219 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ab135-168e-47c9-8fdc-bd4383631d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:07.908719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ab135-168e-47c9-8fdc-bd4383631d76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:07.919873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ab135-168e-47c9-8fdc-bd4383631d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:07.921692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ab135-168e-47c9-8fdc-bd4383631d76 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-5b2ab135-168e-47c9-8fdc-bd4383631d76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:02:07.936440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:07.938265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:02:07.939175 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:02:07.939175 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:07.939175 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:07.941268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 477/959] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:07.952463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b0441-d422-47a3-abf5-a740a2dbd33a None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:08.056861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:08.059786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b0441-d422-47a3-abf5-a740a2dbd33a None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:08.095699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafa79f-afcf-4bdc-823b-c99524530a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:08.096312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafa79f-afcf-4bdc-823b-c99524530a71 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:08.096594 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafa79f-afcf-4bdc-823b-c99524530a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:08.096901 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafa79f-afcf-4bdc-823b-c99524530a71 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:08.097550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafa79f-afcf-4bdc-823b-c99524530a71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:08.097857 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafa79f-afcf-4bdc-823b-c99524530a71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:08.099232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafa79f-afcf-4bdc-823b-c99524530a71 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:08.111895 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 483/960] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:08.117702 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ab135-168e-47c9-8fdc-bd4383631d76 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:08.177671 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:08.226080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ab135-168e-47c9-8fdc-bd4383631d76 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:08.260915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941beaf-f1a0-4e5b-8d12-ec8a17304ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:08.261535 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941beaf-f1a0-4e5b-8d12-ec8a17304ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:08.261813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941beaf-f1a0-4e5b-8d12-ec8a17304ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:08.262076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941beaf-f1a0-4e5b-8d12-ec8a17304ce5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:08.270158 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941beaf-f1a0-4e5b-8d12-ec8a17304ce5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:08.271614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941beaf-f1a0-4e5b-8d12-ec8a17304ce5 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-3941beaf-f1a0-4e5b-8d12-ec8a17304ce5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:02:08.282751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:08.283732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:08.285103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:08.285103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:08.285103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:02:08.285103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:02:08.285103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:02:08.285103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:02:08.285103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:02:08.285103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:02:08.285103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 12 12:02:08.285103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:08.285103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 478/961] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 12 12:02:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:08.291910 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafa79f-afcf-4bdc-823b-c99524530a71 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:08.400825 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafa79f-afcf-4bdc-823b-c99524530a71 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:08.436168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83af4e-256f-48f6-9ccb-f55005854b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:08.436723 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83af4e-256f-48f6-9ccb-f55005854b2b None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:08.437034 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83af4e-256f-48f6-9ccb-f55005854b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:08.437840 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83af4e-256f-48f6-9ccb-f55005854b2b None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:08.438575 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83af4e-256f-48f6-9ccb-f55005854b2b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:08.438878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83af4e-256f-48f6-9ccb-f55005854b2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:08.440311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83af4e-256f-48f6-9ccb-f55005854b2b None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:08.458472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 484/962] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Aug 12 12:02:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:08.464300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941beaf-f1a0-4e5b-8d12-ec8a17304ce5 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:08.481738 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118eeb9-4496-4606-acf7-9a9d14c092c9 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:08.482049 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118eeb9-4496-4606-acf7-9a9d14c092c9 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:08.482310 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118eeb9-4496-4606-acf7-9a9d14c092c9 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:08.484433 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 485/963] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:02:08.489461 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733aedb5-48ad-4bc9-82f0-4197ddb3e9b3 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:08.489747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733aedb5-48ad-4bc9-82f0-4197ddb3e9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:08.490027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733aedb5-48ad-4bc9-82f0-4197ddb3e9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:08.507849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-733aedb5-48ad-4bc9-82f0-4197ddb3e9b3 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:08.509079 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-733aedb5-48ad-4bc9-82f0-4197ddb3e9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:08.523239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:08.582475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941beaf-f1a0-4e5b-8d12-ec8a17304ce5 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:08.591659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 486/964] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:02:08.606281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:08.620696 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e89b2f-7a45-45d2-bf42-a2a2532eafde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:08.621307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e89b2f-7a45-45d2-bf42-a2a2532eafde None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:08.621587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e89b2f-7a45-45d2-bf42-a2a2532eafde None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:08.621867 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e89b2f-7a45-45d2-bf42-a2a2532eafde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:08.696918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e89b2f-7a45-45d2-bf42-a2a2532eafde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:08.698281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e89b2f-7a45-45d2-bf42-a2a2532eafde None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:08.798234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 479/965] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:02:08 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:02:08.807136 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733aedb5-48ad-4bc9-82f0-4197ddb3e9b3 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:08.880723 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e89b2f-7a45-45d2-bf42-a2a2532eafde None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:08.918080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:08.919018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733aedb5-48ad-4bc9-82f0-4197ddb3e9b3 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:08.955551 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1c6b5-5394-4af9-b3ef-53d23df104a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:08.956110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1c6b5-5394-4af9-b3ef-53d23df104a6 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:08.956388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1c6b5-5394-4af9-b3ef-53d23df104a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:08.956682 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1c6b5-5394-4af9-b3ef-53d23df104a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:08.967372 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1c6b5-5394-4af9-b3ef-53d23df104a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:08.969088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1c6b5-5394-4af9-b3ef-53d23df104a6 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:08.977886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-b2c1c6b5-5394-4af9-b3ef-53d23df104a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:02:08.984882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:08.989419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:02:08.990314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:02:08.990314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:08.990314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:08.990314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 487/966] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:08.992067 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1c6b5-5394-4af9-b3ef-53d23df104a6 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:08.993312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e89b2f-7a45-45d2-bf42-a2a2532eafde None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:09.032252 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987397db-6466-4526-8409-06e0b2fd9c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:09.032914 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987397db-6466-4526-8409-06e0b2fd9c78 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:09.033226 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987397db-6466-4526-8409-06e0b2fd9c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:09.034235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987397db-6466-4526-8409-06e0b2fd9c78 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:09.034235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987397db-6466-4526-8409-06e0b2fd9c78 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:09.035509 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987397db-6466-4526-8409-06e0b2fd9c78 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-987397db-6466-4526-8409-06e0b2fd9c78 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:09.045044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:09.046076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:09.047434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:09.047434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:09.047434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:09.047434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:09.047434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:09.047434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:09.047434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:09.047434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:09.047434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:09.047434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:09.047434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:02:09.047434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:09.047434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 12:02:09.047434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:09.047434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 480/967] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 12 12:02:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:02:09.053093 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987397db-6466-4526-8409-06e0b2fd9c78 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:09.128492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1c6b5-5394-4af9-b3ef-53d23df104a6 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:09.153432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:09.169952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987397db-6466-4526-8409-06e0b2fd9c78 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:09.179907 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bffd7-53b7-431d-9e0e-89c530ce7a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:09.181019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bffd7-53b7-431d-9e0e-89c530ce7a75 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:09.181403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bffd7-53b7-431d-9e0e-89c530ce7a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:09.181792 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bffd7-53b7-431d-9e0e-89c530ce7a75 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:09.182844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bffd7-53b7-431d-9e0e-89c530ce7a75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:09.183247 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bffd7-53b7-431d-9e0e-89c530ce7a75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:09.185134 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bffd7-53b7-431d-9e0e-89c530ce7a75 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:09.201377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 488/968] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:09.207428 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bffd7-53b7-431d-9e0e-89c530ce7a75 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:09.218904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2fd4b-68a7-4cb5-ba6a-b720233f4ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:09.220281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2fd4b-68a7-4cb5-ba6a-b720233f4ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:09.221108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2fd4b-68a7-4cb5-ba6a-b720233f4ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:09.221652 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2fd4b-68a7-4cb5-ba6a-b720233f4ef9 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:09.222730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d2fd4b-68a7-4cb5-ba6a-b720233f4ef9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:09.223758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d2fd4b-68a7-4cb5-ba6a-b720233f4ef9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:09.225949 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d2fd4b-68a7-4cb5-ba6a-b720233f4ef9 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:09.233583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 481/969] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 12 12:02:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:09.239320 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2fd4b-68a7-4cb5-ba6a-b720233f4ef9 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:09.272649 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:09.314539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bffd7-53b7-431d-9e0e-89c530ce7a75 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:09.349719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114bedf-5078-400d-94e7-9ddee904541e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:09.349940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2fd4b-68a7-4cb5-ba6a-b720233f4ef9 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:09.350363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114bedf-5078-400d-94e7-9ddee904541e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:09.350637 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114bedf-5078-400d-94e7-9ddee904541e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:09.350912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114bedf-5078-400d-94e7-9ddee904541e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:09.362593 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114bedf-5078-400d-94e7-9ddee904541e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:09.364037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114bedf-5078-400d-94e7-9ddee904541e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:09.375956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-f114bedf-5078-400d-94e7-9ddee904541e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:02:09.375956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:09.375956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:09.375956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:09.375956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:09.375956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:09.375956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:09.375956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:09.375956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:09.375956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:09.375956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:09.375956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:09.375956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:02:09.375956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:02:09.375956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:09.377121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:09.378026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:02:09.378026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:02:09.378026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:02:09.378026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:02:09.378026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:02:09.378026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:02:09.378026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:02:09.378026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:09.378026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 489/970] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:02:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:09.378509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d55b2-6abb-4b7d-84f0-b7adb11213e8 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:09.378509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d55b2-6abb-4b7d-84f0-b7adb11213e8 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:09.379096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d55b2-6abb-4b7d-84f0-b7adb11213e8 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:09.380164 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 490/971] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:09.384742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d55b2-6abb-4b7d-84f0-b7adb11213e8 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:09.390583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c5f59-7fad-4ba0-b247-9ed9f6793308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:09.391189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c5f59-7fad-4ba0-b247-9ed9f6793308 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:09.391462 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c5f59-7fad-4ba0-b247-9ed9f6793308 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:09.391730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c5f59-7fad-4ba0-b247-9ed9f6793308 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:09.392363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c5f59-7fad-4ba0-b247-9ed9f6793308 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:09.392682 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c5f59-7fad-4ba0-b247-9ed9f6793308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:09.394058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c5f59-7fad-4ba0-b247-9ed9f6793308 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:09.403609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 482/972] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 12 12:02:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:09.403609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc7a73-c2f9-4167-8ece-cd6d73e4202d None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:09.403609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc7a73-c2f9-4167-8ece-cd6d73e4202d None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:09.403609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc7a73-c2f9-4167-8ece-cd6d73e4202d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:09.420078 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-6ddc7a73-c2f9-4167-8ece-cd6d73e4202d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:09.421508 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6ddc7a73-c2f9-4167-8ece-cd6d73e4202d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:09.429804 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:09.488665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 483/973] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:09.489460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc7a73-c2f9-4167-8ece-cd6d73e4202d None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:09.502158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d55b2-6abb-4b7d-84f0-b7adb11213e8 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:09.523753 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:09.539221 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00b428-a10e-4636-bb00-35b042c579c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:09.539824 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00b428-a10e-4636-bb00-35b042c579c5 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:09.540104 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00b428-a10e-4636-bb00-35b042c579c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:09.540376 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00b428-a10e-4636-bb00-35b042c579c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:09.541090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00b428-a10e-4636-bb00-35b042c579c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:09.541440 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00b428-a10e-4636-bb00-35b042c579c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:09.542861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00b428-a10e-4636-bb00-35b042c579c5 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:09.560970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 491/974] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:02:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:09.562317 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e2494-a043-4ae2-acdb-e950a53ede02 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:09.562883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e2494-a043-4ae2-acdb-e950a53ede02 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:09.563338 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e2494-a043-4ae2-acdb-e950a53ede02 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:09.569651 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 492/975] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:09.570439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e2494-a043-4ae2-acdb-e950a53ede02 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:09.600755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc7a73-c2f9-4167-8ece-cd6d73e4202d None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:09.641680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b6e84-d1cf-41f2-b749-5943dfe96ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:09.642358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b6e84-d1cf-41f2-b749-5943dfe96ac9 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:09.642642 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b6e84-d1cf-41f2-b749-5943dfe96ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:09.642946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b6e84-d1cf-41f2-b749-5943dfe96ac9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:09.643613 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b6e84-d1cf-41f2-b749-5943dfe96ac9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:09.645005 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b6e84-d1cf-41f2-b749-5943dfe96ac9 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:09.679268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e2494-a043-4ae2-acdb-e950a53ede02 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:09.712690 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293ee52-3cdf-4bca-aee2-0852e405d341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:09.713283 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293ee52-3cdf-4bca-aee2-0852e405d341 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:09.713563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293ee52-3cdf-4bca-aee2-0852e405d341 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:09.713833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293ee52-3cdf-4bca-aee2-0852e405d341 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:09.714576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293ee52-3cdf-4bca-aee2-0852e405d341 None admin] RBAC: Adding query filter params (role.id=ff702c6529bc4a57a4f3bfe2b4973444, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=ab4e815bbe7f4ff880f4cc633da1ed8c) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:09.714880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293ee52-3cdf-4bca-aee2-0852e405d341 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:09.716292 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293ee52-3cdf-4bca-aee2-0852e405d341 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:09.719161 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:09.723123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 493/976] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:02:09 2022] GET /identity/v3/role_assignments?user.id=ab4e815bbe7f4ff880f4cc633da1ed8c&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=ff702c6529bc4a57a4f3bfe2b4973444 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:09.724150 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec91207-33df-4317-b22c-32832bdf4096 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:09.724433 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec91207-33df-4317-b22c-32832bdf4096 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:09.724766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec91207-33df-4317-b22c-32832bdf4096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:09.751908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 484/977] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:02:09 2022] POST /identity/v3/endpoints => generated 349 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:02:09.753950 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-dec91207-33df-4317-b22c-32832bdf4096 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:09.755382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-dec91207-33df-4317-b22c-32832bdf4096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:09.761601 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b6e84-d1cf-41f2-b749-5943dfe96ac9 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:09.837621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:09.878735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 494/978] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:09.914226 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b6e84-d1cf-41f2-b749-5943dfe96ac9 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:09.953162 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8396a2c-0bb4-4766-a30f-36f36e59e47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:09.953916 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8396a2c-0bb4-4766-a30f-36f36e59e47c None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:09.954197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8396a2c-0bb4-4766-a30f-36f36e59e47c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:09.954479 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8396a2c-0bb4-4766-a30f-36f36e59e47c None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:09.955119 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8396a2c-0bb4-4766-a30f-36f36e59e47c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:09.956533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8396a2c-0bb4-4766-a30f-36f36e59e47c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:09.959108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: WARNING keystone.server.flask.common [None req-a8396a2c-0bb4-4766-a30f-36f36e59e47c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 12 12:02:10.004260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.sql.core [None req-a8396a2c-0bb4-4766-a30f-36f36e59e47c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 12 12:02:10.004260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 12 12:02:10.004260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [parameters: {'user_id': 'c00023101ab94d7a8f508009c69eda1c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 12 12:02:10.004260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81793) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 12 12:02:10.021873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-a8396a2c-0bb4-4766-a30f-36f36e59e47c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 12 12:02:10.021873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:10.021873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 12 12:02:10.021873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 12 12:02:10.021873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 12 12:02:10.021873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 12 12:02:10.021873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 12 12:02:10.021873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application result = self._query(query) Aug 12 12:02:10.021873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 12 12:02:10.021873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application conn.query(q) Aug 12 12:02:10.021873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 12 12:02:10.021873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 12 12:02:10.021873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 12 12:02:10.021873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application result.read() Aug 12 12:02:10.021873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 12 12:02:10.022839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application next(self.gen) Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application yield resource Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application next(self.gen) Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self.session.rollback() Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self.force_reraise() Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise self.value Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application session.commit() Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 12 12:02:10.023760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._prepare_impl() Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self.session.flush() Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._flush(objects) Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application compat.raise_( Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application flush_context.execute() Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rec.execute(self) Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 12 12:02:10.024627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 12 12:02:10.027538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application result = self._query(query) Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application conn.query(q) Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application result.read() Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 12 12:02:10.029596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 12 12:02:10.032877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 12 12:02:10.032877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 12 12:02:10.032877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 12 12:02:10.032877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 12 12:02:10.032877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 12 12:02:10.032877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 12 12:02:10.032877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c00023101ab94d7a8f508009c69eda1c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 12 12:02:10.032877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 12 12:02:10.032877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:10.032877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:10.032877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:10.032877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:10.032877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:10.032877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:10.032877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 12 12:02:10.033731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 12 12:02:10.034637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 12 12:02:10.034637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 12 12:02:10.034637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 12 12:02:10.034637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 12 12:02:10.034637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 12 12:02:10.034637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:10.034637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 485/979] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Aug 12 12:02:09 2022] POST /identity/v3/users => generated 160 bytes in 262 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:02:10.035048 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec91207-33df-4317-b22c-32832bdf4096 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:10.140671 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec91207-33df-4317-b22c-32832bdf4096 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:10.164133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8396a2c-0bb4-4766-a30f-36f36e59e47c None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:10.176673 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4591631-163f-4cdd-a49d-ae54325e6b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:10.177296 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4591631-163f-4cdd-a49d-ae54325e6b0d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:10.177858 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4591631-163f-4cdd-a49d-ae54325e6b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:10.178109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4591631-163f-4cdd-a49d-ae54325e6b0d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:10.185576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4591631-163f-4cdd-a49d-ae54325e6b0d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:10.187597 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4591631-163f-4cdd-a49d-ae54325e6b0d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-d4591631-163f-4cdd-a49d-ae54325e6b0d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:02:10.199558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:10.200540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:10.201499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:10.201499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:10.201499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:02:10.201499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:02:10.201499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:02:10.201499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:02:10.201499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:02:10.201499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:02:10.201499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 12 12:02:10.201499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:10.201499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 495/980] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:02:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:10.208194 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4591631-163f-4cdd-a49d-ae54325e6b0d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:10.243551 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:10.256942 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:10.306082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8396a2c-0bb4-4766-a30f-36f36e59e47c None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:10.325650 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4591631-163f-4cdd-a49d-ae54325e6b0d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:10.343900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edd30f-4ece-47cd-8974-ffc0898136d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:10.344261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edd30f-4ece-47cd-8974-ffc0898136d3 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:10.344651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edd30f-4ece-47cd-8974-ffc0898136d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:10.344958 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edd30f-4ece-47cd-8974-ffc0898136d3 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:10.345746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6edd30f-4ece-47cd-8974-ffc0898136d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:10.347222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6edd30f-4ece-47cd-8974-ffc0898136d3 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:10.366085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8d8f1-fcd5-4f3b-99b4-b6f2d70c3a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:10.366826 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8d8f1-fcd5-4f3b-99b4-b6f2d70c3a42 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:10.367175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8d8f1-fcd5-4f3b-99b4-b6f2d70c3a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:10.367563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8d8f1-fcd5-4f3b-99b4-b6f2d70c3a42 None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:10.368309 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b8d8f1-fcd5-4f3b-99b4-b6f2d70c3a42 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:10.368702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b8d8f1-fcd5-4f3b-99b4-b6f2d70c3a42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:10.370192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b8d8f1-fcd5-4f3b-99b4-b6f2d70c3a42 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:10.406335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 496/981] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 12 12:02:10 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:10.419331 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 486/982] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 12 12:02:10 2022] POST /identity/v3/services => generated 242 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:02:11.731975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92572f1-b1ad-4da6-b586-1bcd705d45da None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:11.732524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92572f1-b1ad-4da6-b586-1bcd705d45da None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:11.732524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92572f1-b1ad-4da6-b586-1bcd705d45da None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:11.734425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 497/983] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:11.739576 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11676e-2475-4eea-905b-bfeeefa2814b None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:11.739998 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11676e-2475-4eea-905b-bfeeefa2814b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:11.740183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11676e-2475-4eea-905b-bfeeefa2814b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:11.759062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-be11676e-2475-4eea-905b-bfeeefa2814b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:11.760563 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-be11676e-2475-4eea-905b-bfeeefa2814b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:11.793012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb8ce0-bd58-46fa-badf-845ccc06604d None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:11.793313 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb8ce0-bd58-46fa-badf-845ccc06604d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:11.793589 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb8ce0-bd58-46fa-badf-845ccc06604d None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:11.795105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 498/984] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:02:11.802345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be4164-90e4-4805-9376-eb0134778bbf None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:11.802345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be4164-90e4-4805-9376-eb0134778bbf None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:11.802345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be4164-90e4-4805-9376-eb0134778bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:11.821508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-a1be4164-90e4-4805-9376-eb0134778bbf None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:11.822714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a1be4164-90e4-4805-9376-eb0134778bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:11.852676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 487/985] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:11.893935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 499/986] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:02:12.117472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11676e-2475-4eea-905b-bfeeefa2814b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:12.164754 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be4164-90e4-4805-9376-eb0134778bbf None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:12.166304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:12.184689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:12.224321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11676e-2475-4eea-905b-bfeeefa2814b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:12.261571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c9784-ec11-4e21-9a09-76a7a453f9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:12.262204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c9784-ec11-4e21-9a09-76a7a453f9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:12.262486 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c9784-ec11-4e21-9a09-76a7a453f9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:12.262760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c9784-ec11-4e21-9a09-76a7a453f9f9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:12.263349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c9784-ec11-4e21-9a09-76a7a453f9f9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:12.265298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c9784-ec11-4e21-9a09-76a7a453f9f9 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:12.273939 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be4164-90e4-4805-9376-eb0134778bbf None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-bc8c9784-ec11-4e21-9a09-76a7a453f9f9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:12.275905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:12.276893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:12.277725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:12.277725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:12.277725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:12.277725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:12.277725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:12.277725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:12.277725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:12.277725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:12.277725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:12.277725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:12.277725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:02:12.277725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:12.277725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 12:02:12.277725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:12.277725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 488/987] 10.222.0.27 () {56 vars in 1105 bytes} [Fri Aug 12 12:02:12 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:12.283538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c9784-ec11-4e21-9a09-76a7a453f9f9 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:12.315828 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4aa74-d7d5-48d8-9b49-be3103df9edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:12.316745 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4aa74-d7d5-48d8-9b49-be3103df9edf None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:12.317171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4aa74-d7d5-48d8-9b49-be3103df9edf None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:12.317586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4aa74-d7d5-48d8-9b49-be3103df9edf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:12.318423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4aa74-d7d5-48d8-9b49-be3103df9edf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:12.320077 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4aa74-d7d5-48d8-9b49-be3103df9edf None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-d4c4aa74-d7d5-48d8-9b49-be3103df9edf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:12.330394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:12.331585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:12.332528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:12.332528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:12.332528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:12.332528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:12.332528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:12.332528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:12.332528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:12.332528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:12.332528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:12.332528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:12.332528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:02:12.332528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:12.332528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 12:02:12.332528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:12.335212 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 500/988] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 12 12:02:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:02:12.339610 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d54122-8ae5-4e3c-86d0-7b1f3f9a7369 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:12.339905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d54122-8ae5-4e3c-86d0-7b1f3f9a7369 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:12.340184 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d54122-8ae5-4e3c-86d0-7b1f3f9a7369 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:12.341598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 501/989] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:02:12.342235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d54122-8ae5-4e3c-86d0-7b1f3f9a7369 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:12.390103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c9784-ec11-4e21-9a09-76a7a453f9f9 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:12.399879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:12.423340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75825a-372e-4467-b332-9122fe88d34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:12.423946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75825a-372e-4467-b332-9122fe88d34f None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:12.424227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75825a-372e-4467-b332-9122fe88d34f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:12.424498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75825a-372e-4467-b332-9122fe88d34f None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:12.425164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75825a-372e-4467-b332-9122fe88d34f None admin] RBAC: Adding query filter params (name=image) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:12.425467 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75825a-372e-4467-b332-9122fe88d34f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:12.426826 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75825a-372e-4467-b332-9122fe88d34f None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:12.429290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:12.432665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 489/990] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:12.434048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5a270-f179-41f7-8271-befa87055a02 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:12.434510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5a270-f179-41f7-8271-befa87055a02 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:12.434957 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5a270-f179-41f7-8271-befa87055a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:12.448482 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d54122-8ae5-4e3c-86d0-7b1f3f9a7369 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:12.452562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-f8a5a270-f179-41f7-8271-befa87055a02 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:12.453784 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f8a5a270-f179-41f7-8271-befa87055a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:12.483309 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4b3db-3000-4b32-85ea-c78bc14b40bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:12.484544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4b3db-3000-4b32-85ea-c78bc14b40bf None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:12.484876 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4b3db-3000-4b32-85ea-c78bc14b40bf None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:12.485151 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4b3db-3000-4b32-85ea-c78bc14b40bf None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:12.485784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4b3db-3000-4b32-85ea-c78bc14b40bf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:12.486088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4b3db-3000-4b32-85ea-c78bc14b40bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:12.487447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4b3db-3000-4b32-85ea-c78bc14b40bf None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:12.492091 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 502/991] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 12 12:02:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:12.492899 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4b3db-3000-4b32-85ea-c78bc14b40bf None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:12.523662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 490/992] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:12.524493 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5a270-f179-41f7-8271-befa87055a02 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:12.595275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4b3db-3000-4b32-85ea-c78bc14b40bf None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:12.628168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b717ee6-652f-4728-b184-dc41b8244055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:12.628936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b717ee6-652f-4728-b184-dc41b8244055 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:12.629087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5a270-f179-41f7-8271-befa87055a02 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:12.629272 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b717ee6-652f-4728-b184-dc41b8244055 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:12.629496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b717ee6-652f-4728-b184-dc41b8244055 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:12.630120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b717ee6-652f-4728-b184-dc41b8244055 None admin] RBAC: Adding query filter params (type=image) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:12.630425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b717ee6-652f-4728-b184-dc41b8244055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:12.631569 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:12.631805 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b717ee6-652f-4728-b184-dc41b8244055 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:12.636460 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 503/993] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:12.680254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba26c93-4744-4cdd-a10b-39631fbdb077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:12.680907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba26c93-4744-4cdd-a10b-39631fbdb077 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:12.681188 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba26c93-4744-4cdd-a10b-39631fbdb077 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:12.681461 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba26c93-4744-4cdd-a10b-39631fbdb077 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:12.682106 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba26c93-4744-4cdd-a10b-39631fbdb077 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:12.682413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba26c93-4744-4cdd-a10b-39631fbdb077 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:12.683856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba26c93-4744-4cdd-a10b-39631fbdb077 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:12.690267 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 491/994] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 12 12:02:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:12.696402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b717ee6-652f-4728-b184-dc41b8244055 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:12.709756 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:12.808289 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba26c93-4744-4cdd-a10b-39631fbdb077 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:12.811218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b717ee6-652f-4728-b184-dc41b8244055 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:12.851181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3ea57-5734-40f4-ba50-1ab0707f3b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:12.851923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3ea57-5734-40f4-ba50-1ab0707f3b8c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:12.852281 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3ea57-5734-40f4-ba50-1ab0707f3b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:12.852488 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3ea57-5734-40f4-ba50-1ab0707f3b8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:12.853223 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3ea57-5734-40f4-ba50-1ab0707f3b8c None admin] RBAC: Adding query filter params (interface=admin, service_id=b6a8123264e74a1fbfd27ee231001170, region_id=RegionOne) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:12.853533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3ea57-5734-40f4-ba50-1ab0707f3b8c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:12.854895 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3ea57-5734-40f4-ba50-1ab0707f3b8c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:12.865295 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 504/995] 10.222.0.27 () {56 vars in 1249 bytes} [Fri Aug 12 12:02:12 2022] GET /identity/v3/endpoints?service_id=b6a8123264e74a1fbfd27ee231001170&interface=admin®ion_id=RegionOne => generated 193 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:12.865952 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3ea57-5734-40f4-ba50-1ab0707f3b8c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:12.913383 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba26c93-4744-4cdd-a10b-39631fbdb077 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:12.948393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:12.949290 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e47e1-c54c-4538-8439-cf52b47f2967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:12.949924 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e47e1-c54c-4538-8439-cf52b47f2967 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:12.950217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e47e1-c54c-4538-8439-cf52b47f2967 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:12.950489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e47e1-c54c-4538-8439-cf52b47f2967 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:12.951192 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e47e1-c54c-4538-8439-cf52b47f2967 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:12.953084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e47e1-c54c-4538-8439-cf52b47f2967 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-227e47e1-c54c-4538-8439-cf52b47f2967 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:12.964683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:12.965733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:12.966635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:12.966635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:12.966635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:12.966635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:12.966635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:12.966635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:12.966635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:12.966635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:12.966635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:12.966635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:12.966635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:02:12.966635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:12.966635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 12:02:12.966635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:12.966635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 492/996] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:02:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:02:12.972552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e47e1-c54c-4538-8439-cf52b47f2967 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:12.977635 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3ea57-5734-40f4-ba50-1ab0707f3b8c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:13.005148 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:13.011660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9281895-e7a5-4322-baa8-6fd990a6d927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:13.016927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9281895-e7a5-4322-baa8-6fd990a6d927 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:13.016927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9281895-e7a5-4322-baa8-6fd990a6d927 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:13.016927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9281895-e7a5-4322-baa8-6fd990a6d927 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:13.016927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9281895-e7a5-4322-baa8-6fd990a6d927 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:13.016927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9281895-e7a5-4322-baa8-6fd990a6d927 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:13.022562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 505/997] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 12 12:02:12 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:02:13.084184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e47e1-c54c-4538-8439-cf52b47f2967 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:13.122295 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfcd1a2-0fac-4286-91ae-f0c532bf8ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:13.122908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfcd1a2-0fac-4286-91ae-f0c532bf8ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:13.123187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfcd1a2-0fac-4286-91ae-f0c532bf8ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:13.123459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfcd1a2-0fac-4286-91ae-f0c532bf8ed8 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:13.124623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfcd1a2-0fac-4286-91ae-f0c532bf8ed8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:13.124954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfcd1a2-0fac-4286-91ae-f0c532bf8ed8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:13.126535 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfcd1a2-0fac-4286-91ae-f0c532bf8ed8 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:13.132105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 493/998] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 12 12:02:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:13.138352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9281895-e7a5-4322-baa8-6fd990a6d927 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:13.244930 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9281895-e7a5-4322-baa8-6fd990a6d927 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:13.279795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378ec27-bed2-417d-b725-c5436b6db0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:13.280998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378ec27-bed2-417d-b725-c5436b6db0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:13.281333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378ec27-bed2-417d-b725-c5436b6db0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:13.281687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378ec27-bed2-417d-b725-c5436b6db0d2 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:13.282325 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378ec27-bed2-417d-b725-c5436b6db0d2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:13.282631 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378ec27-bed2-417d-b725-c5436b6db0d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:13.284008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378ec27-bed2-417d-b725-c5436b6db0d2 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:13.289148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 506/999] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 12 12:02:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:13.295132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfcd1a2-0fac-4286-91ae-f0c532bf8ed8 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:13.392410 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:13.398550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfcd1a2-0fac-4286-91ae-f0c532bf8ed8 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:13.432872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a9d6d-be86-427e-b552-c770683ccb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:13.433525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a9d6d-be86-427e-b552-c770683ccb20 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:13.433802 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a9d6d-be86-427e-b552-c770683ccb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:13.434069 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a9d6d-be86-427e-b552-c770683ccb20 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:13.434800 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a9d6d-be86-427e-b552-c770683ccb20 None admin] RBAC: Adding query filter params (interface=public, service_id=1f66f5a0d81d4881aa4e502bff7a86a8, region_id=RegionOne) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:13.435106 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a9d6d-be86-427e-b552-c770683ccb20 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:13.436455 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a9d6d-be86-427e-b552-c770683ccb20 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:13.445971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 494/1000] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Aug 12 12:02:13 2022] GET /identity/v3/endpoints?service_id=1f66f5a0d81d4881aa4e502bff7a86a8&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:13.447523 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378ec27-bed2-417d-b725-c5436b6db0d2 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:13.450395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:13.554141 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378ec27-bed2-417d-b725-c5436b6db0d2 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:13.589346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f5f0d-2ce3-42f3-aea3-43e227856af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:13.590294 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f5f0d-2ce3-42f3-aea3-43e227856af3 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:13.590806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f5f0d-2ce3-42f3-aea3-43e227856af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:13.591286 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f5f0d-2ce3-42f3-aea3-43e227856af3 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:13.592095 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f5f0d-2ce3-42f3-aea3-43e227856af3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:13.594292 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f5f0d-2ce3-42f3-aea3-43e227856af3 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:13.604755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 507/1001] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 12 12:02:13 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:02:14.559254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f13fc-242a-4ff4-86c8-81ecfd467490 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:14.559781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f13fc-242a-4ff4-86c8-81ecfd467490 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:14.559858 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f13fc-242a-4ff4-86c8-81ecfd467490 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:14.561353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 495/1002] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:14.566769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6aa172-e557-47ca-bd90-dca29c45bf16 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:14.567183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6aa172-e557-47ca-bd90-dca29c45bf16 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:14.567354 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6aa172-e557-47ca-bd90-dca29c45bf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:14.586850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-4b6aa172-e557-47ca-bd90-dca29c45bf16 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:14.588038 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4b6aa172-e557-47ca-bd90-dca29c45bf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:14.648543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:14.657160 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 508/1003] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:14.924471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f13fc-242a-4ff4-86c8-81ecfd467490 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:14.938869 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2a13f-fadd-4b01-8b5c-417eaafd02d7 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:14.939155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2a13f-fadd-4b01-8b5c-417eaafd02d7 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:14.939561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2a13f-fadd-4b01-8b5c-417eaafd02d7 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:14.940919 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 509/1004] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:02:14.945590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6af94b-9045-47c4-b705-23f86f9475d2 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:14.945877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6af94b-9045-47c4-b705-23f86f9475d2 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:14.946320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6af94b-9045-47c4-b705-23f86f9475d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:14.964463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-dc6af94b-9045-47c4-b705-23f86f9475d2 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:14.965705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-dc6af94b-9045-47c4-b705-23f86f9475d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:15.032125 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f13fc-242a-4ff4-86c8-81ecfd467490 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:15.037252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 510/1005] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:02:15.068256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36838272-fe2f-46aa-8e73-10b48dcd31e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:15.068890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36838272-fe2f-46aa-8e73-10b48dcd31e6 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:15.069175 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36838272-fe2f-46aa-8e73-10b48dcd31e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:15.069447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36838272-fe2f-46aa-8e73-10b48dcd31e6 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:15.070101 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36838272-fe2f-46aa-8e73-10b48dcd31e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:15.071459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36838272-fe2f-46aa-8e73-10b48dcd31e6 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:15.160685 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 496/1006] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 12 12:02:14 2022] POST /identity/v3/services => generated 242 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:02:15.306620 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6af94b-9045-47c4-b705-23f86f9475d2 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:15.409110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6af94b-9045-47c4-b705-23f86f9475d2 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:15.435524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:15.449959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecef24-b192-460e-a013-cfccaeb63731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:15.458887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecef24-b192-460e-a013-cfccaeb63731 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:15.458887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecef24-b192-460e-a013-cfccaeb63731 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:15.458887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecef24-b192-460e-a013-cfccaeb63731 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:15.458887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecef24-b192-460e-a013-cfccaeb63731 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:15.458887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecef24-b192-460e-a013-cfccaeb63731 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-58ecef24-b192-460e-a013-cfccaeb63731 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:15.461547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:15.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:15.463352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:15.463352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:15.463352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:15.463352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:15.463352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:15.463352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:15.463352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:15.463352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:15.463352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:15.463352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:15.463352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:02:15.463352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:15.463352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 12 12:02:15.463352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:15.463352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 511/1007] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 12 12:02:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:15.470715 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36838272-fe2f-46aa-8e73-10b48dcd31e6 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:15.494712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:15.543986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96358396-3711-4822-80c3-d25a9c617906 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:15.544306 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96358396-3711-4822-80c3-d25a9c617906 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:15.544586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96358396-3711-4822-80c3-d25a9c617906 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:15.546675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 512/1008] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:02:15.551857 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f689572-25e4-4071-82f6-be74f3e54af8 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:15.552142 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f689572-25e4-4071-82f6-be74f3e54af8 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:15.552693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f689572-25e4-4071-82f6-be74f3e54af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:15.570881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-3f689572-25e4-4071-82f6-be74f3e54af8 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:15.571987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-3f689572-25e4-4071-82f6-be74f3e54af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:15.577999 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36838272-fe2f-46aa-8e73-10b48dcd31e6 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:15.615698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d34c0-f5b1-4154-a693-888710c8b16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:15.616299 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d34c0-f5b1-4154-a693-888710c8b16a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:15.616576 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d34c0-f5b1-4154-a693-888710c8b16a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:15.616875 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d34c0-f5b1-4154-a693-888710c8b16a None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:15.617502 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d34c0-f5b1-4154-a693-888710c8b16a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:15.617991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d34c0-f5b1-4154-a693-888710c8b16a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:15.619151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d34c0-f5b1-4154-a693-888710c8b16a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:15.624870 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 497/1009] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 12 12:02:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:15.630411 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d34c0-f5b1-4154-a693-888710c8b16a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:15.649207 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 513/1010] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:15 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:02:15.718022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:15.733853 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d34c0-f5b1-4154-a693-888710c8b16a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:15.767253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefa6cc-42d1-4831-97f0-fc7aac70ae16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:15.767850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefa6cc-42d1-4831-97f0-fc7aac70ae16 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:15.768130 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefa6cc-42d1-4831-97f0-fc7aac70ae16 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:15.768399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefa6cc-42d1-4831-97f0-fc7aac70ae16 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:15.769055 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefa6cc-42d1-4831-97f0-fc7aac70ae16 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:15.769359 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefa6cc-42d1-4831-97f0-fc7aac70ae16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:15.770710 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefa6cc-42d1-4831-97f0-fc7aac70ae16 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:15.780931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 498/1011] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 12 12:02:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:15.782458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f689572-25e4-4071-82f6-be74f3e54af8 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:15.893991 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f689572-25e4-4071-82f6-be74f3e54af8 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:15.910659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:15.923032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefa6cc-42d1-4831-97f0-fc7aac70ae16 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:15.929918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bda59-6a82-4d0b-ad32-5bab54b11bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:15.930530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bda59-6a82-4d0b-ad32-5bab54b11bee None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:15.930814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bda59-6a82-4d0b-ad32-5bab54b11bee None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:15.931088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90bda59-6a82-4d0b-ad32-5bab54b11bee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:15.931762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bda59-6a82-4d0b-ad32-5bab54b11bee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:15.933590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90bda59-6a82-4d0b-ad32-5bab54b11bee None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:15.998097 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 514/1012] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:02:15 2022] POST /identity/v3/endpoints => generated 325 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:02:16.029962 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefa6cc-42d1-4831-97f0-fc7aac70ae16 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:16.077793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cbb77-8e3f-4989-86ee-88b54d6d6e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:16.078433 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cbb77-8e3f-4989-86ee-88b54d6d6e88 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:16.078714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cbb77-8e3f-4989-86ee-88b54d6d6e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:16.078982 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cbb77-8e3f-4989-86ee-88b54d6d6e88 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:16.079623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96cbb77-8e3f-4989-86ee-88b54d6d6e88 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:16.081028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96cbb77-8e3f-4989-86ee-88b54d6d6e88 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-b96cbb77-8e3f-4989-86ee-88b54d6d6e88 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:16.090727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:16.091792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:16.092765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:16.092765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:16.092765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:16.092765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:16.092765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:16.092765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:16.092765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:16.092765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:16.092765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:16.092765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:16.092765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:02:16.092765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:16.092765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 12 12:02:16.092765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:16.097011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 499/1013] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:02:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:02:16.098604 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bda59-6a82-4d0b-ad32-5bab54b11bee None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:16.210472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90bda59-6a82-4d0b-ad32-5bab54b11bee None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:16.244732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216ad55-0355-4a3d-ae51-91c9e2239595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:16.245342 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216ad55-0355-4a3d-ae51-91c9e2239595 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:16.245625 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216ad55-0355-4a3d-ae51-91c9e2239595 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:16.245899 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216ad55-0355-4a3d-ae51-91c9e2239595 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:16.246536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216ad55-0355-4a3d-ae51-91c9e2239595 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:16.246844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216ad55-0355-4a3d-ae51-91c9e2239595 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:16.248229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216ad55-0355-4a3d-ae51-91c9e2239595 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:16.254709 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 515/1014] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 12 12:02:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:16.260857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cbb77-8e3f-4989-86ee-88b54d6d6e88 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:16.273245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:16.366000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cbb77-8e3f-4989-86ee-88b54d6d6e88 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:16.401126 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c4fcf-b4d5-43e7-b674-d28b6c7aaea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:16.401830 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c4fcf-b4d5-43e7-b674-d28b6c7aaea0 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:16.402110 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c4fcf-b4d5-43e7-b674-d28b6c7aaea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:16.402382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c4fcf-b4d5-43e7-b674-d28b6c7aaea0 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:16.403007 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c4fcf-b4d5-43e7-b674-d28b6c7aaea0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:16.403305 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c4fcf-b4d5-43e7-b674-d28b6c7aaea0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:16.404674 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c4fcf-b4d5-43e7-b674-d28b6c7aaea0 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:16.410230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 500/1015] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 12 12:02:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:16.415148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216ad55-0355-4a3d-ae51-91c9e2239595 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:16.460786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:16.519021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216ad55-0355-4a3d-ae51-91c9e2239595 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:16.552945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10db66a-b09e-4e1f-b466-4a3013144f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:16.553612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10db66a-b09e-4e1f-b466-4a3013144f6a None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:16.553897 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10db66a-b09e-4e1f-b466-4a3013144f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:16.554173 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10db66a-b09e-4e1f-b466-4a3013144f6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:16.554829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10db66a-b09e-4e1f-b466-4a3013144f6a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:16.556197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10db66a-b09e-4e1f-b466-4a3013144f6a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:16.655910 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 516/1016] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:02:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:02:17.095377 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b97da-ac35-458c-a2a3-9142670d5bd6 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:17.095924 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b97da-ac35-458c-a2a3-9142670d5bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:17.095924 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b97da-ac35-458c-a2a3-9142670d5bd6 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:17.097265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 501/1017] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:17.103439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d5543-8d13-4134-a7a9-363ede7b609c None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:17.103735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d5543-8d13-4134-a7a9-363ede7b609c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:17.103995 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d5543-8d13-4134-a7a9-363ede7b609c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:17.122002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-5a1d5543-8d13-4134-a7a9-363ede7b609c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:17.123192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-5a1d5543-8d13-4134-a7a9-363ede7b609c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:17.196310 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 517/1018] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:17.462085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b97da-ac35-458c-a2a3-9142670d5bd6 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:17.533136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:17.564328 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b97da-ac35-458c-a2a3-9142670d5bd6 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:17.599435 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759e9db-2eba-4d24-b624-a7ca8717cf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:17.600066 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759e9db-2eba-4d24-b624-a7ca8717cf2b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:17.600344 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759e9db-2eba-4d24-b624-a7ca8717cf2b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:17.600616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759e9db-2eba-4d24-b624-a7ca8717cf2b None admin] PATH_INFO: `/v3/services/image` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:17.601228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759e9db-2eba-4d24-b624-a7ca8717cf2b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:17.602585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759e9db-2eba-4d24-b624-a7ca8717cf2b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-a759e9db-2eba-4d24-b624-a7ca8717cf2b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:17.611325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:17.612285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:17.613565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:17.613565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:17.613565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:17.613565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:17.613565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:17.613565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:17.613565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:17.613565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:17.613565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:17.613565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:17.613565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:02:17.613565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:17.613565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 12:02:17.613565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:17.616722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 502/1019] 10.222.0.27 () {56 vars in 1105 bytes} [Fri Aug 12 12:02:17 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:17.618639 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d5543-8d13-4134-a7a9-363ede7b609c None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:17.643514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:17.721651 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d5543-8d13-4134-a7a9-363ede7b609c None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:17.756898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12feb47d-1555-47fa-90a4-acf0bc0d81fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:17.757838 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12feb47d-1555-47fa-90a4-acf0bc0d81fc None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:17.758262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12feb47d-1555-47fa-90a4-acf0bc0d81fc None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:17.758674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12feb47d-1555-47fa-90a4-acf0bc0d81fc None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:17.759457 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12feb47d-1555-47fa-90a4-acf0bc0d81fc None admin] RBAC: Adding query filter params (name=image) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:17.759899 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12feb47d-1555-47fa-90a4-acf0bc0d81fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:17.761478 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12feb47d-1555-47fa-90a4-acf0bc0d81fc None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:17.776747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 518/1020] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:17.782294 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759e9db-2eba-4d24-b624-a7ca8717cf2b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:17.890523 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759e9db-2eba-4d24-b624-a7ca8717cf2b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:17.923806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74eb00-6433-4476-b07a-ba357a0501f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:17.924407 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74eb00-6433-4476-b07a-ba357a0501f3 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:17.924714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74eb00-6433-4476-b07a-ba357a0501f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:17.924990 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74eb00-6433-4476-b07a-ba357a0501f3 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:17.925610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74eb00-6433-4476-b07a-ba357a0501f3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:17.925909 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74eb00-6433-4476-b07a-ba357a0501f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:17.927510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74eb00-6433-4476-b07a-ba357a0501f3 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:17.930486 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:17.933969 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 503/1021] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:17.940827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12feb47d-1555-47fa-90a4-acf0bc0d81fc None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:17.949143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc34d16-1e65-47da-a9d5-13adbbe14425 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:17.949143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc34d16-1e65-47da-a9d5-13adbbe14425 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:17.949143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc34d16-1e65-47da-a9d5-13adbbe14425 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:17.950249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 504/1022] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:02:17.956263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f7a11-1fb6-4a48-a6fe-7ca1878c94b5 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:17.956544 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f7a11-1fb6-4a48-a6fe-7ca1878c94b5 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:17.956843 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f7a11-1fb6-4a48-a6fe-7ca1878c94b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:17.975513 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-720f7a11-1fb6-4a48-a6fe-7ca1878c94b5 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:17.976619 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-720f7a11-1fb6-4a48-a6fe-7ca1878c94b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:18.031494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:18.046860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 505/1023] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:02:18.047161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12feb47d-1555-47fa-90a4-acf0bc0d81fc None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:18.082269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8430e2-0d40-4f86-affb-8b215db51f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:18.082876 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8430e2-0d40-4f86-affb-8b215db51f07 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:18.083160 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8430e2-0d40-4f86-affb-8b215db51f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:18.083432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8430e2-0d40-4f86-affb-8b215db51f07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:18.084135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8430e2-0d40-4f86-affb-8b215db51f07 None admin] RBAC: Adding query filter params (interface=public, service_id=b5e1f1d845f94e77a9199953e7a0e79d, region_id=RegionOne) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:18.084448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8430e2-0d40-4f86-affb-8b215db51f07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:18.085842 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8430e2-0d40-4f86-affb-8b215db51f07 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:18.091068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 519/1024] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Aug 12 12:02:17 2022] GET /identity/v3/endpoints?service_id=b5e1f1d845f94e77a9199953e7a0e79d&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:18.096666 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f7a11-1fb6-4a48-a6fe-7ca1878c94b5 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:18.194808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f7a11-1fb6-4a48-a6fe-7ca1878c94b5 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:18.223405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:18.235363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c76ec8-eee1-445d-8abb-c8b3ed15b49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:18.235363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c76ec8-eee1-445d-8abb-c8b3ed15b49b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:18.235363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c76ec8-eee1-445d-8abb-c8b3ed15b49b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:18.235363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c76ec8-eee1-445d-8abb-c8b3ed15b49b None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:18.235800 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c76ec8-eee1-445d-8abb-c8b3ed15b49b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:18.235800 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c76ec8-eee1-445d-8abb-c8b3ed15b49b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:18.249466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 506/1025] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 12 12:02:18 2022] GET /identity/v3/services => generated 1957 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:02:18.316127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8430e2-0d40-4f86-affb-8b215db51f07 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:18.426512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8430e2-0d40-4f86-affb-8b215db51f07 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:18.462593 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097cfad-a1a3-49d7-a766-54371c1eb3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:18.463229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097cfad-a1a3-49d7-a766-54371c1eb3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:18.463515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097cfad-a1a3-49d7-a766-54371c1eb3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:18.463791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097cfad-a1a3-49d7-a766-54371c1eb3a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:18.464936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097cfad-a1a3-49d7-a766-54371c1eb3a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:18.466379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097cfad-a1a3-49d7-a766-54371c1eb3a9 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:18.468672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 520/1026] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:18.473748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c76ec8-eee1-445d-8abb-c8b3ed15b49b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:18.574851 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c76ec8-eee1-445d-8abb-c8b3ed15b49b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:18.608448 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b1906-38ec-46b0-911f-e2a0eb4213e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:18.609143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b1906-38ec-46b0-911f-e2a0eb4213e6 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:18.609424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b1906-38ec-46b0-911f-e2a0eb4213e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:18.609695 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b1906-38ec-46b0-911f-e2a0eb4213e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:18.610322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b1906-38ec-46b0-911f-e2a0eb4213e6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:18.611736 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b1906-38ec-46b0-911f-e2a0eb4213e6 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:18.667005 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473662c-c5dc-46ec-85ee-02eec27c4937 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:18.667399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473662c-c5dc-46ec-85ee-02eec27c4937 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:18.667740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473662c-c5dc-46ec-85ee-02eec27c4937 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:18.669190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 521/1027] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:02:18.674813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e20f8c-4673-4cd8-bff1-b7db754cd439 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:18.675095 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e20f8c-4673-4cd8-bff1-b7db754cd439 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:18.675370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e20f8c-4673-4cd8-bff1-b7db754cd439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:18.693611 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-64e20f8c-4673-4cd8-bff1-b7db754cd439 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:18.694779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-64e20f8c-4673-4cd8-bff1-b7db754cd439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:18.712668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 507/1028] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 12 12:02:18 2022] POST /identity/v3/projects => generated 311 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:02:18.712905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:18.770916 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 522/1029] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:02:19.037139 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b1906-38ec-46b0-911f-e2a0eb4213e6 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:19.086613 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:19.142470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b1906-38ec-46b0-911f-e2a0eb4213e6 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:19.177118 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78de0b6-30a5-41c9-a1d5-47837834cce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:19.177991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78de0b6-30a5-41c9-a1d5-47837834cce0 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:19.178269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78de0b6-30a5-41c9-a1d5-47837834cce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:19.178538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78de0b6-30a5-41c9-a1d5-47837834cce0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:19.189100 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78de0b6-30a5-41c9-a1d5-47837834cce0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-f78de0b6-30a5-41c9-a1d5-47837834cce0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:02:19.194261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:19.195144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:19.195965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:19.195965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:19.195965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:19.195965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:19.195965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:02:19.195965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:02:19.195965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:02:19.195965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:02:19.195965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:02:19.195965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:02:19.195965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 12:02:19.195965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:19.195965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 508/1030] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Aug 12 12:02:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:19.202780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e20f8c-4673-4cd8-bff1-b7db754cd439 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:19.304405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e20f8c-4673-4cd8-bff1-b7db754cd439 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:19.339980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec218d6-36b0-4ac5-8143-72108153b410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:19.340616 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec218d6-36b0-4ac5-8143-72108153b410 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:19.340924 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec218d6-36b0-4ac5-8143-72108153b410 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:19.341199 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec218d6-36b0-4ac5-8143-72108153b410 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:19.341879 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec218d6-36b0-4ac5-8143-72108153b410 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:19.342739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec218d6-36b0-4ac5-8143-72108153b410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:19.344115 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec218d6-36b0-4ac5-8143-72108153b410 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:19.357002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 523/1031] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Aug 12 12:02:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:19.359321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78de0b6-30a5-41c9-a1d5-47837834cce0 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:19.459641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78de0b6-30a5-41c9-a1d5-47837834cce0 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:19.466808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:19.494242 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c487a9f-f1b6-4bab-be43-200f30749008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:19.528587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c487a9f-f1b6-4bab-be43-200f30749008 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:19.528587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c487a9f-f1b6-4bab-be43-200f30749008 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:19.528587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c487a9f-f1b6-4bab-be43-200f30749008 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:19.528587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c487a9f-f1b6-4bab-be43-200f30749008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:19.528587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c487a9f-f1b6-4bab-be43-200f30749008 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:19.528587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-1c487a9f-f1b6-4bab-be43-200f30749008 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:19.528587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:19.529109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:19.530189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:19.531234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 509/1032] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:19.531302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec218d6-36b0-4ac5-8143-72108153b410 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:19.542385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:19.632059 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec218d6-36b0-4ac5-8143-72108153b410 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:19.665368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995a3f2-32ac-4e9e-a6a1-a934a5fe72dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:19.665975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995a3f2-32ac-4e9e-a6a1-a934a5fe72dc None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:19.666258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995a3f2-32ac-4e9e-a6a1-a934a5fe72dc None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:19.666534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995a3f2-32ac-4e9e-a6a1-a934a5fe72dc None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:19.667176 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995a3f2-32ac-4e9e-a6a1-a934a5fe72dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:19.667482 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995a3f2-32ac-4e9e-a6a1-a934a5fe72dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:19.668883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995a3f2-32ac-4e9e-a6a1-a934a5fe72dc None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:19.686770 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 524/1033] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:19.687930 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c487a9f-f1b6-4bab-be43-200f30749008 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:19.789160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c487a9f-f1b6-4bab-be43-200f30749008 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:19.829979 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263adb1-eb94-4c0f-8d5a-c9c0e308a43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:19.830662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263adb1-eb94-4c0f-8d5a-c9c0e308a43f None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:19.830942 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263adb1-eb94-4c0f-8d5a-c9c0e308a43f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:19.831211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263adb1-eb94-4c0f-8d5a-c9c0e308a43f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:19.839433 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263adb1-eb94-4c0f-8d5a-c9c0e308a43f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:19.841001 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263adb1-eb94-4c0f-8d5a-c9c0e308a43f None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-c263adb1-eb94-4c0f-8d5a-c9c0e308a43f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:02:19.853490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:19.854468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:19.861090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:19.861090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:19.861090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:02:19.861090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:02:19.861090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:02:19.861090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:02:19.861090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:02:19.861090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:02:19.861090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:02:19.861090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:19.861090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 510/1034] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 12 12:02:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:19.861680 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995a3f2-32ac-4e9e-a6a1-a934a5fe72dc None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:19.934598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:19.964509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995a3f2-32ac-4e9e-a6a1-a934a5fe72dc None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:19.999289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f4f7a-fd0c-4dfa-a033-0edad6a83f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:19.999956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f4f7a-fd0c-4dfa-a033-0edad6a83f6a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:20.000227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f4f7a-fd0c-4dfa-a033-0edad6a83f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:20.000502 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f4f7a-fd0c-4dfa-a033-0edad6a83f6a None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:20.001264 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f4f7a-fd0c-4dfa-a033-0edad6a83f6a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:20.001567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f4f7a-fd0c-4dfa-a033-0edad6a83f6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:20.003024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f4f7a-fd0c-4dfa-a033-0edad6a83f6a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:20.020691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 525/1035] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 12 12:02:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:20.026227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263adb1-eb94-4c0f-8d5a-c9c0e308a43f None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:20.048185 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:20.132683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263adb1-eb94-4c0f-8d5a-c9c0e308a43f None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:20.152673 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3af63a-194b-4bfc-8676-39b31239a8a1 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:20.152972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3af63a-194b-4bfc-8676-39b31239a8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:20.153681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3af63a-194b-4bfc-8676-39b31239a8a1 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:20.155767 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 526/1036] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:02:20.161339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd1d4f-0ab7-4d05-8b08-3f67ff518cde None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:20.161645 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd1d4f-0ab7-4d05-8b08-3f67ff518cde None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:20.161886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd1d4f-0ab7-4d05-8b08-3f67ff518cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:20.168455 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b0911-6189-4537-9d4d-0921051be736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:20.169917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b0911-6189-4537-9d4d-0921051be736 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:20.170237 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b0911-6189-4537-9d4d-0921051be736 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:20.171095 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b0911-6189-4537-9d4d-0921051be736 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:20.181816 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-a8cd1d4f-0ab7-4d05-8b08-3f67ff518cde None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:20.183354 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a8cd1d4f-0ab7-4d05-8b08-3f67ff518cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:20.244698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b0911-6189-4537-9d4d-0921051be736 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:20.246661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b0911-6189-4537-9d4d-0921051be736 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:20.250723 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 527/1037] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:02:20.266394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:20.354882 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 511/1038] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:02:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:02:20.356112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd1d4f-0ab7-4d05-8b08-3f67ff518cde None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:20.414869 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:20.457928 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd1d4f-0ab7-4d05-8b08-3f67ff518cde None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:20.491405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e59ff7-3a20-48ed-9634-e70877cf4625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:20.492033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e59ff7-3a20-48ed-9634-e70877cf4625 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:20.492314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e59ff7-3a20-48ed-9634-e70877cf4625 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:20.492586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e59ff7-3a20-48ed-9634-e70877cf4625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:20.503250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e59ff7-3a20-48ed-9634-e70877cf4625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:20.504775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e59ff7-3a20-48ed-9634-e70877cf4625 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-64e59ff7-3a20-48ed-9634-e70877cf4625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:02:20.519923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:20.525659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:02:20.526579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:02:20.526579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:20.526579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:20.526809 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b0911-6189-4537-9d4d-0921051be736 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:20.526886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 528/1039] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:20.526886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e59ff7-3a20-48ed-9634-e70877cf4625 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:20.626221 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b0911-6189-4537-9d4d-0921051be736 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:20.629459 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e59ff7-3a20-48ed-9634-e70877cf4625 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:20.639865 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:20.644682 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:20.662331 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab65894-63e2-4483-8310-a7828bba91e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:20.663040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab65894-63e2-4483-8310-a7828bba91e9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:20.663329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab65894-63e2-4483-8310-a7828bba91e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:20.663750 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab65894-63e2-4483-8310-a7828bba91e9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:20.664108 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30547f9f-9901-4f52-ac1d-18057b15d66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:20.664329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab65894-63e2-4483-8310-a7828bba91e9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:20.664735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30547f9f-9901-4f52-ac1d-18057b15d66e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:20.667092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30547f9f-9901-4f52-ac1d-18057b15d66e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:20.667092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30547f9f-9901-4f52-ac1d-18057b15d66e None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:20.667258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab65894-63e2-4483-8310-a7828bba91e9 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:20.667329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30547f9f-9901-4f52-ac1d-18057b15d66e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:20.667329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30547f9f-9901-4f52-ac1d-18057b15d66e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:20.667505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30547f9f-9901-4f52-ac1d-18057b15d66e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-8ab65894-63e2-4483-8310-a7828bba91e9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:20.675153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:20.676227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:20.677465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:20.677465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:20.677465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:20.677465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:20.677465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:20.677465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:20.677465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:20.677465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:20.677465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:20.677465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:20.677465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:02:20.677465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:02:20.677465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 12 12:02:20.677465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:20.678990 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 512/1040] 10.222.0.27 () {56 vars in 1105 bytes} [Fri Aug 12 12:02:20 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:02:20.681246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 529/1041] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:20.684753 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab65894-63e2-4483-8310-a7828bba91e9 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:20.687069 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30547f9f-9901-4f52-ac1d-18057b15d66e None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:20.783002 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab65894-63e2-4483-8310-a7828bba91e9 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:20.785515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30547f9f-9901-4f52-ac1d-18057b15d66e None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:20.819559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e88c1-fd49-4e8a-8b5a-c5589188a768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:20.820186 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e88c1-fd49-4e8a-8b5a-c5589188a768 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:20.820466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e88c1-fd49-4e8a-8b5a-c5589188a768 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:20.820833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e88c1-fd49-4e8a-8b5a-c5589188a768 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:20.822695 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08ec47-4d15-40ea-a57d-e6de8b2d67d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:20.823288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08ec47-4d15-40ea-a57d-e6de8b2d67d7 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:20.823563 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08ec47-4d15-40ea-a57d-e6de8b2d67d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:20.823832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08ec47-4d15-40ea-a57d-e6de8b2d67d7 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:20.824450 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08ec47-4d15-40ea-a57d-e6de8b2d67d7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:20.824777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08ec47-4d15-40ea-a57d-e6de8b2d67d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:20.826120 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08ec47-4d15-40ea-a57d-e6de8b2d67d7 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:20.827587 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e88c1-fd49-4e8a-8b5a-c5589188a768 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:20.829047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e88c1-fd49-4e8a-8b5a-c5589188a768 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:20.830989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 513/1042] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:20.832114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21481673-8a8f-4824-bf8b-1183ad9cacee None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:20.832396 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21481673-8a8f-4824-bf8b-1183ad9cacee None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:20.832687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21481673-8a8f-4824-bf8b-1183ad9cacee None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:20.833862 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 514/1043] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:20.836455 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21481673-8a8f-4824-bf8b-1183ad9cacee None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:20.846127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-bf8e88c1-fd49-4e8a-8b5a-c5589188a768 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:02:20.846127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:20.846127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:20.846127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:20.846127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:20.846127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:20.846127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:20.846127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:20.846127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:20.846127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:20.846127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:20.846127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:20.846127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:02:20.846127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:02:20.846127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:20.847285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:20.848998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:02:20.848998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:02:20.848998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:02:20.848998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:02:20.848998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:02:20.848998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:02:20.848998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:02:20.848998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:20.848998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 530/1044] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:02:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:02:20.848998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60041f48-6718-43be-872e-7b4e18529f14 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:20.848998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60041f48-6718-43be-872e-7b4e18529f14 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:20.849594 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60041f48-6718-43be-872e-7b4e18529f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:20.868996 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-60041f48-6718-43be-872e-7b4e18529f14 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:20.870190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-60041f48-6718-43be-872e-7b4e18529f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:20.885257 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:20.894187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:20.935373 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 531/1045] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:20.936880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60041f48-6718-43be-872e-7b4e18529f14 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:20.941634 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21481673-8a8f-4824-bf8b-1183ad9cacee None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:20.981150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629ddde-21c2-4109-b5c6-ffe554b286e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:20.981761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629ddde-21c2-4109-b5c6-ffe554b286e0 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:20.982048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629ddde-21c2-4109-b5c6-ffe554b286e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:20.982324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629ddde-21c2-4109-b5c6-ffe554b286e0 None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:20.983782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629ddde-21c2-4109-b5c6-ffe554b286e0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:20.984311 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629ddde-21c2-4109-b5c6-ffe554b286e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:20.985707 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629ddde-21c2-4109-b5c6-ffe554b286e0 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:20.995153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 515/1046] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:20.996195 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629ddde-21c2-4109-b5c6-ffe554b286e0 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:21.048694 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60041f48-6718-43be-872e-7b4e18529f14 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:21.088626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611ca3c-572f-4822-b9ec-107de4f2cd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:21.089262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611ca3c-572f-4822-b9ec-107de4f2cd26 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:21.089550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611ca3c-572f-4822-b9ec-107de4f2cd26 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:21.089825 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611ca3c-572f-4822-b9ec-107de4f2cd26 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:21.090517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c611ca3c-572f-4822-b9ec-107de4f2cd26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:21.091328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c611ca3c-572f-4822-b9ec-107de4f2cd26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:21.094008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c611ca3c-572f-4822-b9ec-107de4f2cd26 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:21.104070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629ddde-21c2-4109-b5c6-ffe554b286e0 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:21.109778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 532/1047] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:02:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:21.115694 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611ca3c-572f-4822-b9ec-107de4f2cd26 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:21.115835 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:21.139806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b7095-9bb2-41fa-b2ef-61081fc8cc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:21.140270 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:21.140511 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b7095-9bb2-41fa-b2ef-61081fc8cc74 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:21.140828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b7095-9bb2-41fa-b2ef-61081fc8cc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:21.141205 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b7095-9bb2-41fa-b2ef-61081fc8cc74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:21.141928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b7095-9bb2-41fa-b2ef-61081fc8cc74 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:21.143345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b7095-9bb2-41fa-b2ef-61081fc8cc74 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:21.219340 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c611ca3c-572f-4822-b9ec-107de4f2cd26 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:21.228723 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 516/1048] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:02:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:02:21.228723 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b7095-9bb2-41fa-b2ef-61081fc8cc74 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:21.269264 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489f5bc-1efd-49fc-916e-7df2a9afb2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:21.269887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489f5bc-1efd-49fc-916e-7df2a9afb2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:21.270172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489f5bc-1efd-49fc-916e-7df2a9afb2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:21.270453 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489f5bc-1efd-49fc-916e-7df2a9afb2d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:21.271209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489f5bc-1efd-49fc-916e-7df2a9afb2d5 None admin] RBAC: Adding query filter params (role.id=ff702c6529bc4a57a4f3bfe2b4973444, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=687d0375abd44d2481f988c6df90fad1) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:21.271524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489f5bc-1efd-49fc-916e-7df2a9afb2d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:21.273049 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489f5bc-1efd-49fc-916e-7df2a9afb2d5 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:21.279538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 533/1049] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:02:21 2022] GET /identity/v3/role_assignments?user.id=687d0375abd44d2481f988c6df90fad1&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=ff702c6529bc4a57a4f3bfe2b4973444 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:21.342409 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b7095-9bb2-41fa-b2ef-61081fc8cc74 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:21.382778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663981c0-ec43-44d3-add1-67eadcdd4cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:21.383486 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663981c0-ec43-44d3-add1-67eadcdd4cdc None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:21.383771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663981c0-ec43-44d3-add1-67eadcdd4cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:21.384044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663981c0-ec43-44d3-add1-67eadcdd4cdc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:21.385244 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663981c0-ec43-44d3-add1-67eadcdd4cdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:21.386712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663981c0-ec43-44d3-add1-67eadcdd4cdc None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:21.388679 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 517/1050] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:21.394493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489f5bc-1efd-49fc-916e-7df2a9afb2d5 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:21.488654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:21.507883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489f5bc-1efd-49fc-916e-7df2a9afb2d5 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:21.542499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49319364-5a92-4ce8-80ac-cdff2e77e393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:21.543103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49319364-5a92-4ce8-80ac-cdff2e77e393 None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:21.543387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49319364-5a92-4ce8-80ac-cdff2e77e393 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:21.543661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49319364-5a92-4ce8-80ac-cdff2e77e393 None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:21.544300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49319364-5a92-4ce8-80ac-cdff2e77e393 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:21.545783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49319364-5a92-4ce8-80ac-cdff2e77e393 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:21.660667 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 534/1051] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:02:21 2022] POST /identity/v3/users => generated 285 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:02:23.157814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae4b25-f996-493c-bdcb-314d16d10ca5 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:23.158315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae4b25-f996-493c-bdcb-314d16d10ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:23.158392 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae4b25-f996-493c-bdcb-314d16d10ca5 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:23.159837 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 518/1052] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:23.164819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffa2f6-bf02-4af3-9ab2-c3633536bfcd None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:23.165206 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffa2f6-bf02-4af3-9ab2-c3633536bfcd None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:23.165385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffa2f6-bf02-4af3-9ab2-c3633536bfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:23.183509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-2dffa2f6-bf02-4af3-9ab2-c3633536bfcd None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:23.184766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2dffa2f6-bf02-4af3-9ab2-c3633536bfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:23.211093 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764b5f6-f9cf-45e8-b913-1d674ba96dc3 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:23.211369 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764b5f6-f9cf-45e8-b913-1d674ba96dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:23.211640 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764b5f6-f9cf-45e8-b913-1d674ba96dc3 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:23.216500 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 519/1053] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:02:23.218504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c05d7-063b-46b6-982e-481461dc2300 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:23.218782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c05d7-063b-46b6-982e-481461dc2300 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:23.219051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c05d7-063b-46b6-982e-481461dc2300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:23.237201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-f06c05d7-063b-46b6-982e-481461dc2300 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:23.238377 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f06c05d7-063b-46b6-982e-481461dc2300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:23.264957 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 535/1054] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:23.265168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:23.307560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 520/1055] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:23.535228 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dffa2f6-bf02-4af3-9ab2-c3633536bfcd None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:23.577879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c05d7-063b-46b6-982e-481461dc2300 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:23.603795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:23.640184 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dffa2f6-bf02-4af3-9ab2-c3633536bfcd None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:23.675688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a1ad8-da26-44c5-9be2-ebe118144f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:23.676322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a1ad8-da26-44c5-9be2-ebe118144f36 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:23.676606 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a1ad8-da26-44c5-9be2-ebe118144f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:23.677154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a1ad8-da26-44c5-9be2-ebe118144f36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:23.679855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c05d7-063b-46b6-982e-481461dc2300 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:23.717715 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759a0dd-3086-48de-bfd5-14da79fb3f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:23.718426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759a0dd-3086-48de-bfd5-14da79fb3f20 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:23.718704 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759a0dd-3086-48de-bfd5-14da79fb3f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:23.718980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759a0dd-3086-48de-bfd5-14da79fb3f20 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:23.728908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759a0dd-3086-48de-bfd5-14da79fb3f20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-d759a0dd-3086-48de-bfd5-14da79fb3f20 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:02:23.734394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:23.735313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:23.736366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:23.736366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:23.736366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:23.736366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:23.736366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:02:23.736366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:02:23.736366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:02:23.736366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:02:23.736366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:02:23.736366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:02:23.736366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 12:02:23.736366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:23.736366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 521/1056] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Aug 12 12:02:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:02:23.737315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26921ced-da4e-4311-8090-47c5c6a3eede None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:23.737315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26921ced-da4e-4311-8090-47c5c6a3eede None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:23.737315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26921ced-da4e-4311-8090-47c5c6a3eede None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:23.740682 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 522/1057] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:23.741288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26921ced-da4e-4311-8090-47c5c6a3eede None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:23.751180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a1ad8-da26-44c5-9be2-ebe118144f36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:23.752676 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a1ad8-da26-44c5-9be2-ebe118144f36 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:23.766944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:23.820540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:23.846946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26921ced-da4e-4311-8090-47c5c6a3eede None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:23.855329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 536/1058] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:02:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:02:23.856567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683ba57-a305-478c-8be5-7fc2cf6d730d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:23.856889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683ba57-a305-478c-8be5-7fc2cf6d730d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:23.857148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683ba57-a305-478c-8be5-7fc2cf6d730d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:23.874823 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-0683ba57-a305-478c-8be5-7fc2cf6d730d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:23.876011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0683ba57-a305-478c-8be5-7fc2cf6d730d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:23.881029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c40072-a027-4bd2-acf3-755ad12e2a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:23.881624 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c40072-a027-4bd2-acf3-755ad12e2a6b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:23.881898 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c40072-a027-4bd2-acf3-755ad12e2a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:23.882166 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c40072-a027-4bd2-acf3-755ad12e2a6b None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:23.882818 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c40072-a027-4bd2-acf3-755ad12e2a6b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:23.883121 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c40072-a027-4bd2-acf3-755ad12e2a6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:23.884467 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c40072-a027-4bd2-acf3-755ad12e2a6b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:23.893015 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 523/1059] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Aug 12 12:02:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:23.893809 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c40072-a027-4bd2-acf3-755ad12e2a6b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:23.942829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 537/1060] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:23.943733 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683ba57-a305-478c-8be5-7fc2cf6d730d None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:23.994505 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:24.013150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c40072-a027-4bd2-acf3-755ad12e2a6b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:24.044682 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:24.051071 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683ba57-a305-478c-8be5-7fc2cf6d730d None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:24.054800 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b542c28-de70-4915-8515-ee0c8456e250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:24.055426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b542c28-de70-4915-8515-ee0c8456e250 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:24.055706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b542c28-de70-4915-8515-ee0c8456e250 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:24.055974 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b542c28-de70-4915-8515-ee0c8456e250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:24.067394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b542c28-de70-4915-8515-ee0c8456e250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:24.068790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b542c28-de70-4915-8515-ee0c8456e250 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-0b542c28-de70-4915-8515-ee0c8456e250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:02:24.084028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:24.090372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:02:24.091334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:02:24.091334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:24.091334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:24.091334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 524/1061] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:24.091334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b542c28-de70-4915-8515-ee0c8456e250 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:24.094231 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf6767-e24c-4061-8273-f4c9cee007f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:24.101835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf6767-e24c-4061-8273-f4c9cee007f0 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:24.101835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf6767-e24c-4061-8273-f4c9cee007f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:24.101835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf6767-e24c-4061-8273-f4c9cee007f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:24.106151 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf6767-e24c-4061-8273-f4c9cee007f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:24.107643 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf6767-e24c-4061-8273-f4c9cee007f0 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-efcf6767-e24c-4061-8273-f4c9cee007f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:02:24.122393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:24.123759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:02:24.125042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:02:24.125042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:24.125042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:24.125042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 538/1062] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:24.129428 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf6767-e24c-4061-8273-f4c9cee007f0 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:24.201975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b542c28-de70-4915-8515-ee0c8456e250 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:24.237613 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a1f8b-2715-47b9-ab19-2e90c67e0053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:24.238199 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf6767-e24c-4061-8273-f4c9cee007f0 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:24.238647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a1f8b-2715-47b9-ab19-2e90c67e0053 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:24.238647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a1f8b-2715-47b9-ab19-2e90c67e0053 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:24.238831 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a1f8b-2715-47b9-ab19-2e90c67e0053 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:24.239349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a1f8b-2715-47b9-ab19-2e90c67e0053 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:24.239658 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a1f8b-2715-47b9-ab19-2e90c67e0053 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:24.241110 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a1f8b-2715-47b9-ab19-2e90c67e0053 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:24.248916 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:24.253675 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 525/1063] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:24.254465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a1f8b-2715-47b9-ab19-2e90c67e0053 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:24.274544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f5bb1-6342-4aa6-bcc5-f46901c6ac32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:24.275167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f5bb1-6342-4aa6-bcc5-f46901c6ac32 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:24.275445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f5bb1-6342-4aa6-bcc5-f46901c6ac32 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:24.275718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f5bb1-6342-4aa6-bcc5-f46901c6ac32 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:24.276376 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f5bb1-6342-4aa6-bcc5-f46901c6ac32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:24.276778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f5bb1-6342-4aa6-bcc5-f46901c6ac32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:24.278217 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f5bb1-6342-4aa6-bcc5-f46901c6ac32 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:24.292491 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 539/1064] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:24.298987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f5bb1-6342-4aa6-bcc5-f46901c6ac32 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:24.311777 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:24.421258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f5bb1-6342-4aa6-bcc5-f46901c6ac32 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:24.423944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a1f8b-2715-47b9-ab19-2e90c67e0053 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:24.459975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcbbd4-f24e-4f40-99b2-57a196dddc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:24.460855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcbbd4-f24e-4f40-99b2-57a196dddc18 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:24.461239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcbbd4-f24e-4f40-99b2-57a196dddc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:24.461518 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcbbd4-f24e-4f40-99b2-57a196dddc18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:24.464005 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391c453-73a3-4d39-8c04-72a735a1c01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:24.464621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391c453-73a3-4d39-8c04-72a735a1c01f None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:24.464934 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391c453-73a3-4d39-8c04-72a735a1c01f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:24.465207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391c453-73a3-4d39-8c04-72a735a1c01f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:24.469991 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcbbd4-f24e-4f40-99b2-57a196dddc18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:24.471035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcbbd4-f24e-4f40-99b2-57a196dddc18 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:24.476707 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391c453-73a3-4d39-8c04-72a735a1c01f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-3391c453-73a3-4d39-8c04-72a735a1c01f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:02:24.481770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:24.483325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:24.484276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:24.484276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:24.484276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:24.484276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:24.484276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:02:24.484276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:02:24.484276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:02:24.484276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:02:24.484276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:02:24.484276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:02:24.484276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:02:24.484276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-3ffcbbd4-f24e-4f40-99b2-57a196dddc18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:02:24.485147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:24.485993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:24.486975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:24.486975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:24.486975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:02:24.486975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:02:24.486975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:02:24.486975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:02:24.486975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:02:24.486975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:02:24.486975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:02:24.486975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:24.487507 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 526/1065] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:02:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:24.489917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 540/1066] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 12 12:02:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:02:24.490112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391c453-73a3-4d39-8c04-72a735a1c01f None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:24.490760 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcbbd4-f24e-4f40-99b2-57a196dddc18 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:24.556459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:24.565344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:24.598181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391c453-73a3-4d39-8c04-72a735a1c01f None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:24.599296 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcbbd4-f24e-4f40-99b2-57a196dddc18 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:24.631647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36512f6-9122-4de4-9be3-1287a4351369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:24.632245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36512f6-9122-4de4-9be3-1287a4351369 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:24.632524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36512f6-9122-4de4-9be3-1287a4351369 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:24.632829 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36512f6-9122-4de4-9be3-1287a4351369 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:24.633121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f8ed8-2ac5-4950-a6e6-d5ec75788448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:24.633488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36512f6-9122-4de4-9be3-1287a4351369 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:24.633717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f8ed8-2ac5-4950-a6e6-d5ec75788448 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:24.633805 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36512f6-9122-4de4-9be3-1287a4351369 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:24.633996 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f8ed8-2ac5-4950-a6e6-d5ec75788448 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:24.634268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f8ed8-2ac5-4950-a6e6-d5ec75788448 None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:24.634985 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f8ed8-2ac5-4950-a6e6-d5ec75788448 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:24.635146 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36512f6-9122-4de4-9be3-1287a4351369 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:24.635285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f8ed8-2ac5-4950-a6e6-d5ec75788448 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:24.636711 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f8ed8-2ac5-4950-a6e6-d5ec75788448 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:24.644017 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 527/1067] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:02:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:24.649606 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36512f6-9122-4de4-9be3-1287a4351369 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:24.654715 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 541/1068] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 12 12:02:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:24.660275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f8ed8-2ac5-4950-a6e6-d5ec75788448 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:24.752429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36512f6-9122-4de4-9be3-1287a4351369 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:24.757297 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f8ed8-2ac5-4950-a6e6-d5ec75788448 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:24.772839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:24.782355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:24.786566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e7fe6-dae7-42ec-b1f7-293d99185e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:24.787187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e7fe6-dae7-42ec-b1f7-293d99185e35 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:24.787467 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e7fe6-dae7-42ec-b1f7-293d99185e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:24.787739 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e7fe6-dae7-42ec-b1f7-293d99185e35 None admin] PATH_INFO: `/v3/users/4b7238f8c5c74c9ba9099c32b7d81df1` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:24.790814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12683e17-f07a-4406-a571-db3f756d47a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:24.791509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12683e17-f07a-4406-a571-db3f756d47a9 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:24.791792 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12683e17-f07a-4406-a571-db3f756d47a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:24.792060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12683e17-f07a-4406-a571-db3f756d47a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:24.801724 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e7fe6-dae7-42ec-b1f7-293d99185e35 None admin] RBAC: Authorizing `identity:get_user(user_id=4b7238f8c5c74c9ba9099c32b7d81df1)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:24.802883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12683e17-f07a-4406-a571-db3f756d47a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:24.803419 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e7fe6-dae7-42ec-b1f7-293d99185e35 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:24.804327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12683e17-f07a-4406-a571-db3f756d47a9 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:24.807513 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 528/1069] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:02:24 2022] GET /identity/v3/users/4b7238f8c5c74c9ba9099c32b7d81df1 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:24.811719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e7fe6-dae7-42ec-b1f7-293d99185e35 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-12683e17-f07a-4406-a571-db3f756d47a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:02:24.822477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:24.823568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:02:24.824732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:02:24.824732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:24.824732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:24.824732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 542/1070] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:02:24.829528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12683e17-f07a-4406-a571-db3f756d47a9 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:24.908621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e7fe6-dae7-42ec-b1f7-293d99185e35 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:24.928061 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12683e17-f07a-4406-a571-db3f756d47a9 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:24.942249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e49d7-201e-4360-86fe-c8b5883b4e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:24.942841 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e49d7-201e-4360-86fe-c8b5883b4e4e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:24.943118 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e49d7-201e-4360-86fe-c8b5883b4e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:24.943389 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e49d7-201e-4360-86fe-c8b5883b4e4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:24.961198 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67f679-d96c-46c5-9bbe-99549a1b4987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:24.961790 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67f679-d96c-46c5-9bbe-99549a1b4987 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:24.962069 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67f679-d96c-46c5-9bbe-99549a1b4987 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:24.962342 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67f679-d96c-46c5-9bbe-99549a1b4987 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:24.962979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c67f679-d96c-46c5-9bbe-99549a1b4987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:24.963281 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c67f679-d96c-46c5-9bbe-99549a1b4987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:24.964664 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c67f679-d96c-46c5-9bbe-99549a1b4987 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:24.976628 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 543/1071] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:24.982563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67f679-d96c-46c5-9bbe-99549a1b4987 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:24.993572 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:25.015822 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e49d7-201e-4360-86fe-c8b5883b4e4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:25.017264 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e49d7-201e-4360-86fe-c8b5883b4e4e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:25.022282 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:25.081215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67f679-d96c-46c5-9bbe-99549a1b4987 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:25.114352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 529/1072] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:02:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 12 12:02:25.120174 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f368b-efda-4b85-93c4-d0a297baab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:25.120432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e49d7-201e-4360-86fe-c8b5883b4e4e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:25.120836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f368b-efda-4b85-93c4-d0a297baab73 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:25.121119 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f368b-efda-4b85-93c4-d0a297baab73 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:25.121388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f368b-efda-4b85-93c4-d0a297baab73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:25.128941 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f368b-efda-4b85-93c4-d0a297baab73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:25.130379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f368b-efda-4b85-93c4-d0a297baab73 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:25.141903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-eb1f368b-efda-4b85-93c4-d0a297baab73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:02:25.141903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:25.141903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:25.141903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:25.141903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:25.141903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:25.141903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:25.141903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:25.141903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:25.141903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:25.141903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:25.141903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:25.141903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:02:25.141903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:02:25.141903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:25.142810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:25.143829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:02:25.143829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:02:25.143829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:02:25.143829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:02:25.143829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:02:25.143829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:02:25.143829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:02:25.143829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:25.143829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 544/1073] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:02:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:02:25.148596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f368b-efda-4b85-93c4-d0a297baab73 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:25.198670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:25.220220 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e49d7-201e-4360-86fe-c8b5883b4e4e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:25.243360 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:25.247087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f368b-efda-4b85-93c4-d0a297baab73 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:25.254415 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d6de2-49de-4470-b867-8f60b5d7c4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:25.255033 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d6de2-49de-4470-b867-8f60b5d7c4ab None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:25.257920 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d6de2-49de-4470-b867-8f60b5d7c4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:25.257920 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d6de2-49de-4470-b867-8f60b5d7c4ab None admin] PATH_INFO: `/v3/projects/46f05f061b4542feb4e8f09d6fd141a6` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:25.257920 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d6de2-49de-4470-b867-8f60b5d7c4ab None admin] RBAC: Authorizing `identity:get_project(project_id=46f05f061b4542feb4e8f09d6fd141a6)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:25.258183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d6de2-49de-4470-b867-8f60b5d7c4ab None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:25.259770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 530/1074] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:02:25 2022] GET /identity/v3/projects/46f05f061b4542feb4e8f09d6fd141a6 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:25.266561 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d6de2-49de-4470-b867-8f60b5d7c4ab None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:25.285286 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5f204-d215-4c04-9ee6-298e5a417cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:25.285956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5f204-d215-4c04-9ee6-298e5a417cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:25.286246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5f204-d215-4c04-9ee6-298e5a417cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:25.286523 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5f204-d215-4c04-9ee6-298e5a417cf8 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:25.287224 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5f204-d215-4c04-9ee6-298e5a417cf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:25.287539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5f204-d215-4c04-9ee6-298e5a417cf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:25.288993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5f204-d215-4c04-9ee6-298e5a417cf8 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:25.304339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 545/1075] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:02:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:25.310217 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5f204-d215-4c04-9ee6-298e5a417cf8 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:25.366477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d6de2-49de-4470-b867-8f60b5d7c4ab None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:25.400063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52ab23-ec42-407c-918d-b366f9344c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:25.400677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52ab23-ec42-407c-918d-b366f9344c62 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:25.400961 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52ab23-ec42-407c-918d-b366f9344c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:25.401241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52ab23-ec42-407c-918d-b366f9344c62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:25.401962 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52ab23-ec42-407c-918d-b366f9344c62 None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=46f05f061b4542feb4e8f09d6fd141a6, user.id=4b7238f8c5c74c9ba9099c32b7d81df1) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:25.402296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52ab23-ec42-407c-918d-b366f9344c62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:25.403718 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52ab23-ec42-407c-918d-b366f9344c62 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:25.409548 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 531/1076] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:02:25 2022] GET /identity/v3/role_assignments?user.id=4b7238f8c5c74c9ba9099c32b7d81df1&scope.project.id=46f05f061b4542feb4e8f09d6fd141a6&role.id=906e3a8a6c7540a38f629f2567133347 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:25.424922 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5f204-d215-4c04-9ee6-298e5a417cf8 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:25.448323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe3a9a-57ef-4a55-acc1-c41eec38c9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:25.449023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe3a9a-57ef-4a55-acc1-c41eec38c9c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:25.449302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe3a9a-57ef-4a55-acc1-c41eec38c9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:25.449571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe3a9a-57ef-4a55-acc1-c41eec38c9c2 None admin] PATH_INFO: `/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/687d0375abd44d2481f988c6df90fad1/roles/ff702c6529bc4a57a4f3bfe2b4973444` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:25.452398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe3a9a-57ef-4a55-acc1-c41eec38c9c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=20c5570631a34c4ab3868eb1cd96e345, user_id=687d0375abd44d2481f988c6df90fad1, role_id=ff702c6529bc4a57a4f3bfe2b4973444)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:25.453913 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe3a9a-57ef-4a55-acc1-c41eec38c9c2 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:25.523012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 546/1077] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:02:25 2022] PUT /identity/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/687d0375abd44d2481f988c6df90fad1/roles/ff702c6529bc4a57a4f3bfe2b4973444 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:02:27.278814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3b657-c01e-40aa-8738-42d886f809c5 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:27.279448 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3b657-c01e-40aa-8738-42d886f809c5 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:27.279600 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3b657-c01e-40aa-8738-42d886f809c5 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:27.280981 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 532/1078] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:27.286277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52820034-8f06-4490-a388-a940ba8d6aa7 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:27.286704 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52820034-8f06-4490-a388-a940ba8d6aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:27.286828 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52820034-8f06-4490-a388-a940ba8d6aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:27.305476 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-52820034-8f06-4490-a388-a940ba8d6aa7 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:27.306653 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-52820034-8f06-4490-a388-a940ba8d6aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:27.320285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:27.373907 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 547/1079] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:02:27.423003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebf4e0-9ece-4e4a-8970-a1eb48b8882b None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:27.423286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebf4e0-9ece-4e4a-8970-a1eb48b8882b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:27.423558 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebf4e0-9ece-4e4a-8970-a1eb48b8882b None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:27.424929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 533/1080] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:27.430355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff111368-c99e-4223-823f-2903f04b65aa None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:27.430644 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff111368-c99e-4223-823f-2903f04b65aa None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:27.430922 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff111368-c99e-4223-823f-2903f04b65aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:27.448562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-ff111368-c99e-4223-823f-2903f04b65aa None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:27.449794 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ff111368-c99e-4223-823f-2903f04b65aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:27.514113 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 548/1081] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:27.642956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebf4e0-9ece-4e4a-8970-a1eb48b8882b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:27.655702 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:27.742374 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebf4e0-9ece-4e4a-8970-a1eb48b8882b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:27.774994 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d112b9-e918-487c-9857-cca0707666f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:27.775605 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d112b9-e918-487c-9857-cca0707666f6 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:27.775886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d112b9-e918-487c-9857-cca0707666f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:27.776160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d112b9-e918-487c-9857-cca0707666f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:27.782355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff111368-c99e-4223-823f-2903f04b65aa None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:27.787140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d112b9-e918-487c-9857-cca0707666f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-62d112b9-e918-487c-9857-cca0707666f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:02:27.791789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:27.792806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:27.793633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:27.793633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:27.793633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:27.793633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:27.793633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:02:27.793633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:02:27.793633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:02:27.793633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:02:27.793633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:02:27.793633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:02:27.793633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:02:27.793633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:27.793633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 534/1082] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:02:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:27.799513 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d112b9-e918-487c-9857-cca0707666f6 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:27.863493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:27.869117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:27.884057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff111368-c99e-4223-823f-2903f04b65aa None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:27.904868 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d112b9-e918-487c-9857-cca0707666f6 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:27.918854 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58a46b-3ad7-4608-8e65-d82bba974997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:27.919528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58a46b-3ad7-4608-8e65-d82bba974997 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:27.919813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58a46b-3ad7-4608-8e65-d82bba974997 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:27.920101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58a46b-3ad7-4608-8e65-d82bba974997 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:27.930510 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58a46b-3ad7-4608-8e65-d82bba974997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:02:27.933210 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe80a3e-12a0-4ecf-9bee-b52ac2470f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:27.933801 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe80a3e-12a0-4ecf-9bee-b52ac2470f29 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:27.934105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe80a3e-12a0-4ecf-9bee-b52ac2470f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:27.934376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe80a3e-12a0-4ecf-9bee-b52ac2470f29 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:27.935040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe80a3e-12a0-4ecf-9bee-b52ac2470f29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-1b58a46b-3ad7-4608-8e65-d82bba974997 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:02:27.935208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:27.936230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:27.937109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:27.937109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:27.937109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:27.937109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:27.937109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:02:27.937109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:02:27.937109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:02:27.937109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:02:27.937109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:02:27.937109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:02:27.937109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 12 12:02:27.937109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:27.937732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe80a3e-12a0-4ecf-9bee-b52ac2470f29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:27.937810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 549/1083] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Aug 12 12:02:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:02:27.937879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe80a3e-12a0-4ecf-9bee-b52ac2470f29 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:27.942776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58a46b-3ad7-4608-8e65-d82bba974997 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:27.950143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 535/1084] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:02:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:27.951918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe80a3e-12a0-4ecf-9bee-b52ac2470f29 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:28.041652 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58a46b-3ad7-4608-8e65-d82bba974997 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:28.051941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe80a3e-12a0-4ecf-9bee-b52ac2470f29 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:28.077870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93463fa-3ad4-4170-a1e6-3115c83d2943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:28.078452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93463fa-3ad4-4170-a1e6-3115c83d2943 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:28.078736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93463fa-3ad4-4170-a1e6-3115c83d2943 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:28.079009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93463fa-3ad4-4170-a1e6-3115c83d2943 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:28.079667 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93463fa-3ad4-4170-a1e6-3115c83d2943 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:28.079971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93463fa-3ad4-4170-a1e6-3115c83d2943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:28.080767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:28.081380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93463fa-3ad4-4170-a1e6-3115c83d2943 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:28.085633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d8bfe-6347-4d46-876b-f2eef6957bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:28.091702 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d8bfe-6347-4d46-876b-f2eef6957bfe None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:28.091813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:28.091903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d8bfe-6347-4d46-876b-f2eef6957bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:28.091903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d8bfe-6347-4d46-876b-f2eef6957bfe None admin] PATH_INFO: `/v3/users/4b7238f8c5c74c9ba9099c32b7d81df1` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:28.091903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d8bfe-6347-4d46-876b-f2eef6957bfe None admin] RBAC: Authorizing `identity:get_user(user_id=4b7238f8c5c74c9ba9099c32b7d81df1)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:28.091903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d8bfe-6347-4d46-876b-f2eef6957bfe None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:28.092323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 550/1085] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Aug 12 12:02:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:28.092410 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 536/1086] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:02:27 2022] GET /identity/v3/users/4b7238f8c5c74c9ba9099c32b7d81df1 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:28.096009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93463fa-3ad4-4170-a1e6-3115c83d2943 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:28.096358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d8bfe-6347-4d46-876b-f2eef6957bfe None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:28.195391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93463fa-3ad4-4170-a1e6-3115c83d2943 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:28.197144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d8bfe-6347-4d46-876b-f2eef6957bfe None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:28.228127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866a375-6acc-468e-941d-4977016126f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:28.228767 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866a375-6acc-468e-941d-4977016126f7 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:28.229045 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866a375-6acc-468e-941d-4977016126f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:28.229318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866a375-6acc-468e-941d-4977016126f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:28.229827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653b916-c573-4037-b988-bbb20c6b5695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:28.230445 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653b916-c573-4037-b988-bbb20c6b5695 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:28.230725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653b916-c573-4037-b988-bbb20c6b5695 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:28.231006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653b916-c573-4037-b988-bbb20c6b5695 None admin] PATH_INFO: `/v3/projects/46f05f061b4542feb4e8f09d6fd141a6` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:28.232790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653b916-c573-4037-b988-bbb20c6b5695 None admin] RBAC: Authorizing `identity:get_project(project_id=46f05f061b4542feb4e8f09d6fd141a6)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:28.234165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653b916-c573-4037-b988-bbb20c6b5695 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:28.235786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 537/1087] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:02:28 2022] GET /identity/v3/projects/46f05f061b4542feb4e8f09d6fd141a6 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:28.240389 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7866a375-6acc-468e-941d-4977016126f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:28.241595 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653b916-c573-4037-b988-bbb20c6b5695 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:28.241841 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7866a375-6acc-468e-941d-4977016126f7 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-7866a375-6acc-468e-941d-4977016126f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:02:28.255938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:28.256892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:02:28.257773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:02:28.257773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:28.257773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:28.257773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 551/1088] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:28.262911 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866a375-6acc-468e-941d-4977016126f7 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:28.305958 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:28.314941 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:28.338681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653b916-c573-4037-b988-bbb20c6b5695 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:28.361007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866a375-6acc-468e-941d-4977016126f7 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:28.373373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9223800-45f5-4830-ab99-3109f57c28c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:28.374024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9223800-45f5-4830-ab99-3109f57c28c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:28.374305 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9223800-45f5-4830-ab99-3109f57c28c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:28.374585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9223800-45f5-4830-ab99-3109f57c28c3 None admin] PATH_INFO: `/v3/projects/46f05f061b4542feb4e8f09d6fd141a6/users/4b7238f8c5c74c9ba9099c32b7d81df1/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:28.376496 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9223800-45f5-4830-ab99-3109f57c28c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=46f05f061b4542feb4e8f09d6fd141a6, user_id=4b7238f8c5c74c9ba9099c32b7d81df1, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:28.377955 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9223800-45f5-4830-ab99-3109f57c28c3 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:28.395267 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f7705-103f-4f9d-ae89-8e51ecfa1392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:28.395865 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f7705-103f-4f9d-ae89-8e51ecfa1392 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:28.396146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f7705-103f-4f9d-ae89-8e51ecfa1392 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:28.396418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f7705-103f-4f9d-ae89-8e51ecfa1392 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:28.397134 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f7705-103f-4f9d-ae89-8e51ecfa1392 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:28.397433 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f7705-103f-4f9d-ae89-8e51ecfa1392 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:28.398782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f7705-103f-4f9d-ae89-8e51ecfa1392 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:28.410815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 552/1089] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:28.416330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f7705-103f-4f9d-ae89-8e51ecfa1392 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:28.442075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 538/1090] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:02:28 2022] PUT /identity/v3/projects/46f05f061b4542feb4e8f09d6fd141a6/users/4b7238f8c5c74c9ba9099c32b7d81df1/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:02:28.521341 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f7705-103f-4f9d-ae89-8e51ecfa1392 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:28.535565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:28.557109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbef77-1e75-479d-b508-8a88173f08ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:28.557745 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbef77-1e75-479d-b508-8a88173f08ed None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:28.558019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbef77-1e75-479d-b508-8a88173f08ed None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:28.558288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbef77-1e75-479d-b508-8a88173f08ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:28.566265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bbef77-1e75-479d-b508-8a88173f08ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:28.568189 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bbef77-1e75-479d-b508-8a88173f08ed None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-e7bbef77-1e75-479d-b508-8a88173f08ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:02:28.580632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:28.581660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:28.582661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:28.582661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:28.582661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:02:28.582661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:02:28.582661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:02:28.582661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:02:28.582661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:02:28.582661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:02:28.582661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 12 12:02:28.582661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:28.582661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 553/1091] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 12 12:02:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:02:28.587738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9223800-45f5-4830-ab99-3109f57c28c3 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:28.688993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9223800-45f5-4830-ab99-3109f57c28c3 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:28.714046 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:28.721831 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbf7a7-6867-4384-8c41-34beab8a646c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:28.727219 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbf7a7-6867-4384-8c41-34beab8a646c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:28.727219 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbf7a7-6867-4384-8c41-34beab8a646c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:28.727219 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbf7a7-6867-4384-8c41-34beab8a646c None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:28.727219 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbf7a7-6867-4384-8c41-34beab8a646c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:28.727219 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbf7a7-6867-4384-8c41-34beab8a646c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:28.727219 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbf7a7-6867-4384-8c41-34beab8a646c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:28.743150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 539/1092] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 12 12:02:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:28.749510 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbef77-1e75-479d-b508-8a88173f08ed None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:28.848030 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbef77-1e75-479d-b508-8a88173f08ed None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:28.880701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a21fe8-5beb-471e-b946-b5a06f85b2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:28.881302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a21fe8-5beb-471e-b946-b5a06f85b2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:28.881581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a21fe8-5beb-471e-b946-b5a06f85b2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:28.881854 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a21fe8-5beb-471e-b946-b5a06f85b2a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:28.921861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:28.946779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a21fe8-5beb-471e-b946-b5a06f85b2a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:28.948239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a21fe8-5beb-471e-b946-b5a06f85b2a5 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:29.043715 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 554/1093] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:02:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:02:29.050456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbf7a7-6867-4384-8c41-34beab8a646c None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:29.146605 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbf7a7-6867-4384-8c41-34beab8a646c None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:29.179500 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf153c47-6ae1-446b-aebc-86f1b8e60249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:29.180104 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf153c47-6ae1-446b-aebc-86f1b8e60249 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:29.180381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf153c47-6ae1-446b-aebc-86f1b8e60249 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:29.180673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf153c47-6ae1-446b-aebc-86f1b8e60249 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:29.190363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf153c47-6ae1-446b-aebc-86f1b8e60249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:29.191779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf153c47-6ae1-446b-aebc-86f1b8e60249 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-cf153c47-6ae1-446b-aebc-86f1b8e60249 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:02:29.205657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:29.206758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:02:29.207652 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:02:29.207652 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:02:29.207652 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:29.207652 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 540/1094] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:29.212610 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a21fe8-5beb-471e-b946-b5a06f85b2a5 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:29.280080 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:29.307803 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a21fe8-5beb-471e-b946-b5a06f85b2a5 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:29.341150 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3eed51-dbd5-4fd5-a117-6ae7cd702eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:29.342459 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3eed51-dbd5-4fd5-a117-6ae7cd702eff None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:29.342689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3eed51-dbd5-4fd5-a117-6ae7cd702eff None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:29.342958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3eed51-dbd5-4fd5-a117-6ae7cd702eff None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:29.343591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3eed51-dbd5-4fd5-a117-6ae7cd702eff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:29.343891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3eed51-dbd5-4fd5-a117-6ae7cd702eff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:29.345286 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3eed51-dbd5-4fd5-a117-6ae7cd702eff None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:29.356080 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 555/1095] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:29.361824 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf153c47-6ae1-446b-aebc-86f1b8e60249 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:29.402148 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:29.462834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf153c47-6ae1-446b-aebc-86f1b8e60249 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:29.495473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b3181-103f-4a0c-862e-5700831de687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:29.496104 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b3181-103f-4a0c-862e-5700831de687 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:29.496386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b3181-103f-4a0c-862e-5700831de687 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:29.496689 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b3181-103f-4a0c-862e-5700831de687 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:29.504213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b3181-103f-4a0c-862e-5700831de687 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:29.505741 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b3181-103f-4a0c-862e-5700831de687 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:29.519065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-eb0b3181-103f-4a0c-862e-5700831de687 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:02:29.519065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:29.519065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:29.519065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:29.519065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:29.519065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:29.519065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:29.519065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:29.519065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:29.519065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:29.519065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:29.519065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:29.519065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:02:29.519065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:02:29.519065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:29.520160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:29.521068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:02:29.521068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:02:29.521068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:02:29.521068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:02:29.521068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:02:29.521068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:02:29.521068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:02:29.521068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:29.521068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 541/1096] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:02:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:29.526059 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3eed51-dbd5-4fd5-a117-6ae7cd702eff None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:29.623111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3eed51-dbd5-4fd5-a117-6ae7cd702eff None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:29.650976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:29.656034 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280c2f6-8151-4cb3-a044-08bef9a9aff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:29.656572 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280c2f6-8151-4cb3-a044-08bef9a9aff8 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:29.656878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280c2f6-8151-4cb3-a044-08bef9a9aff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:29.657155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280c2f6-8151-4cb3-a044-08bef9a9aff8 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:29.657842 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280c2f6-8151-4cb3-a044-08bef9a9aff8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:29.658147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280c2f6-8151-4cb3-a044-08bef9a9aff8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:29.659572 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280c2f6-8151-4cb3-a044-08bef9a9aff8 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:29.671079 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 556/1097] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:02:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:29.677429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b3181-103f-4a0c-862e-5700831de687 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:29.771625 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:29.775184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b3181-103f-4a0c-862e-5700831de687 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:29.809503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61163230-f11a-48f9-9919-eb35be87fdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:29.809793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61163230-f11a-48f9-9919-eb35be87fdd2 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:29.810855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61163230-f11a-48f9-9919-eb35be87fdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:29.811190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61163230-f11a-48f9-9919-eb35be87fdd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:29.811975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61163230-f11a-48f9-9919-eb35be87fdd2 None admin] RBAC: Adding query filter params (role.id=ff702c6529bc4a57a4f3bfe2b4973444, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=687d0375abd44d2481f988c6df90fad1) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:29.812281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61163230-f11a-48f9-9919-eb35be87fdd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:29.813719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61163230-f11a-48f9-9919-eb35be87fdd2 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:29.819706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 542/1098] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:02:29 2022] GET /identity/v3/role_assignments?user.id=687d0375abd44d2481f988c6df90fad1&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=ff702c6529bc4a57a4f3bfe2b4973444 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:30.331931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2e7c4-defa-4d2c-8ed7-be993dbbc5e8 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:30.332215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2e7c4-defa-4d2c-8ed7-be993dbbc5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:30.332499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2e7c4-defa-4d2c-8ed7-be993dbbc5e8 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:30.333891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 557/1099] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:30.339328 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b625695-316b-4947-be27-ae35b377cb20 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:30.339619 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b625695-316b-4947-be27-ae35b377cb20 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:30.339893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b625695-316b-4947-be27-ae35b377cb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:30.358842 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-7b625695-316b-4947-be27-ae35b377cb20 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:30.360140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-7b625695-316b-4947-be27-ae35b377cb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:30.428252 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 543/1100] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:30.695805 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2e7c4-defa-4d2c-8ed7-be993dbbc5e8 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:30.792245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2e7c4-defa-4d2c-8ed7-be993dbbc5e8 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:30.826144 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91c69b-54b3-4669-8a8d-3b5e2b2fba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:30.826753 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91c69b-54b3-4669-8a8d-3b5e2b2fba99 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:30.827030 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91c69b-54b3-4669-8a8d-3b5e2b2fba99 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:30.827301 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91c69b-54b3-4669-8a8d-3b5e2b2fba99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:30.837047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91c69b-54b3-4669-8a8d-3b5e2b2fba99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-df91c69b-54b3-4669-8a8d-3b5e2b2fba99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:02:30.841717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:30.842786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:30.843984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:30.843984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:30.843984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:30.843984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:30.843984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:02:30.843984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:02:30.843984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:02:30.843984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:02:30.843984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:02:30.843984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:02:30.843984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:02:30.843984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:30.843984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 558/1101] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:02:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:30.848785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b625695-316b-4947-be27-ae35b377cb20 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:30.935487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:30.943719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b625695-316b-4947-be27-ae35b377cb20 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:30.976075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55e588-daef-4417-860e-44841672eca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:30.976605 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55e588-daef-4417-860e-44841672eca6 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:30.976915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55e588-daef-4417-860e-44841672eca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:30.977272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55e588-daef-4417-860e-44841672eca6 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:30.978052 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55e588-daef-4417-860e-44841672eca6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:30.978347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55e588-daef-4417-860e-44841672eca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:30.979696 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55e588-daef-4417-860e-44841672eca6 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:30.988367 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 544/1102] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:02:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:30.994080 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91c69b-54b3-4669-8a8d-3b5e2b2fba99 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:31.046062 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:31.092913 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91c69b-54b3-4669-8a8d-3b5e2b2fba99 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:31.127214 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5422d-a4c9-44be-a11b-877631c69511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:31.127836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5422d-a4c9-44be-a11b-877631c69511 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:31.128121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5422d-a4c9-44be-a11b-877631c69511 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:31.128399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5422d-a4c9-44be-a11b-877631c69511 None admin] PATH_INFO: `/v3/users/4b7238f8c5c74c9ba9099c32b7d81df1` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:31.129559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5422d-a4c9-44be-a11b-877631c69511 None admin] RBAC: Authorizing `identity:get_user(user_id=4b7238f8c5c74c9ba9099c32b7d81df1)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:31.131020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5422d-a4c9-44be-a11b-877631c69511 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:31.132588 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 559/1103] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:02:30 2022] GET /identity/v3/users/4b7238f8c5c74c9ba9099c32b7d81df1 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:02:31.138478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55e588-daef-4417-860e-44841672eca6 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:31.236090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55e588-daef-4417-860e-44841672eca6 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:31.268844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aad33d-6089-47cc-8fca-8ad4a06e8132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:31.269434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aad33d-6089-47cc-8fca-8ad4a06e8132 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:31.269719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aad33d-6089-47cc-8fca-8ad4a06e8132 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:31.269995 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aad33d-6089-47cc-8fca-8ad4a06e8132 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:31.292470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:31.334789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aad33d-6089-47cc-8fca-8ad4a06e8132 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:31.336251 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aad33d-6089-47cc-8fca-8ad4a06e8132 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:31.431295 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 545/1104] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:02:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:02:31.437167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5422d-a4c9-44be-a11b-877631c69511 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:31.532550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5422d-a4c9-44be-a11b-877631c69511 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:31.552263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:31.565617 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd4de4-7137-4b2c-8d39-c2349783000f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:31.566261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd4de4-7137-4b2c-8d39-c2349783000f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:31.566541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd4de4-7137-4b2c-8d39-c2349783000f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:31.566811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd4de4-7137-4b2c-8d39-c2349783000f None admin] PATH_INFO: `/v3/projects/46f05f061b4542feb4e8f09d6fd141a6` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:31.567917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd4de4-7137-4b2c-8d39-c2349783000f None admin] RBAC: Authorizing `identity:get_project(project_id=46f05f061b4542feb4e8f09d6fd141a6)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:31.569379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd4de4-7137-4b2c-8d39-c2349783000f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:31.570978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 560/1105] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:02:31 2022] GET /identity/v3/projects/46f05f061b4542feb4e8f09d6fd141a6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:31.577677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aad33d-6089-47cc-8fca-8ad4a06e8132 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:31.632184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:31.676772 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aad33d-6089-47cc-8fca-8ad4a06e8132 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:31.710396 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a184e-57e3-4f3b-a695-75541dd61d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:31.710984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a184e-57e3-4f3b-a695-75541dd61d58 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:31.711267 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a184e-57e3-4f3b-a695-75541dd61d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:31.711541 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a184e-57e3-4f3b-a695-75541dd61d58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:31.712262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a184e-57e3-4f3b-a695-75541dd61d58 None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=46f05f061b4542feb4e8f09d6fd141a6, user.id=4b7238f8c5c74c9ba9099c32b7d81df1) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:31.712571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a184e-57e3-4f3b-a695-75541dd61d58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:31.714008 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a184e-57e3-4f3b-a695-75541dd61d58 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:31.719701 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 546/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:02:31 2022] GET /identity/v3/role_assignments?user.id=4b7238f8c5c74c9ba9099c32b7d81df1&scope.project.id=46f05f061b4542feb4e8f09d6fd141a6&role.id=906e3a8a6c7540a38f629f2567133347 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:33.634881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9657f-bc1d-4737-a57d-336a0c197f07 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:33.635444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9657f-bc1d-4737-a57d-336a0c197f07 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:33.635444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9657f-bc1d-4737-a57d-336a0c197f07 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:33.636777 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 561/1107] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:33.642510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b1320-9a7d-42be-a46d-823d7b3ca0cf None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:33.642922 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b1320-9a7d-42be-a46d-823d7b3ca0cf None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:33.643060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b1320-9a7d-42be-a46d-823d7b3ca0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:33.660686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-fc2b1320-9a7d-42be-a46d-823d7b3ca0cf None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:33.661869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-fc2b1320-9a7d-42be-a46d-823d7b3ca0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:33.724978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 547/1108] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:33.999740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9657f-bc1d-4737-a57d-336a0c197f07 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:34.097463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9657f-bc1d-4737-a57d-336a0c197f07 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:34.130060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d9ff5-c3ad-4b10-9735-16fb6f190ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:34.130668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d9ff5-c3ad-4b10-9735-16fb6f190ded None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:34.130949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d9ff5-c3ad-4b10-9735-16fb6f190ded None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:34.131223 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d9ff5-c3ad-4b10-9735-16fb6f190ded None admin] PATH_INFO: `/v3/domains/default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:34.132321 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d9ff5-c3ad-4b10-9735-16fb6f190ded None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:34.133776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d9ff5-c3ad-4b10-9735-16fb6f190ded None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:34.135310 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 562/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:34.141078 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b1320-9a7d-42be-a46d-823d7b3ca0cf None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:34.189246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:34.238797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b1320-9a7d-42be-a46d-823d7b3ca0cf None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:34.270626 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89842f6d-37da-4586-86cc-6a0afc1beda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:34.271155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89842f6d-37da-4586-86cc-6a0afc1beda6 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:34.271430 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89842f6d-37da-4586-86cc-6a0afc1beda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:34.271699 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89842f6d-37da-4586-86cc-6a0afc1beda6 None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:34.272337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89842f6d-37da-4586-86cc-6a0afc1beda6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:34.273779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89842f6d-37da-4586-86cc-6a0afc1beda6 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:34.407312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 548/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:02:34 2022] POST /identity/v3/users => generated 286 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:02:36.331838 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b44f3-e9c7-4ac6-9920-a30b2ab480fe None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:36.332427 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b44f3-e9c7-4ac6-9920-a30b2ab480fe None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:36.332427 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b44f3-e9c7-4ac6-9920-a30b2ab480fe None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:36.333740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 563/1111] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:36.338990 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a021d-6012-4f11-9abf-2b4dd52fbde7 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:36.339420 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a021d-6012-4f11-9abf-2b4dd52fbde7 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:36.339546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a021d-6012-4f11-9abf-2b4dd52fbde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:36.358211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-223a021d-6012-4f11-9abf-2b4dd52fbde7 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:36.359381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-223a021d-6012-4f11-9abf-2b4dd52fbde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:36.423551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 549/1112] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:36.692584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b44f3-e9c7-4ac6-9920-a30b2ab480fe None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:36.748863 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:36.789650 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b44f3-e9c7-4ac6-9920-a30b2ab480fe None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:36.824275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727d093-eb4d-48b5-8f8c-80420ca0f749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:36.824926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727d093-eb4d-48b5-8f8c-80420ca0f749 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:36.825207 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727d093-eb4d-48b5-8f8c-80420ca0f749 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:36.825478 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727d093-eb4d-48b5-8f8c-80420ca0f749 None admin] PATH_INFO: `/v3/roles/b5e411fc96434df69c6cc1df951b3e6f` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:36.826858 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727d093-eb4d-48b5-8f8c-80420ca0f749 None admin] RBAC: Authorizing `identity:get_role(role_id=b5e411fc96434df69c6cc1df951b3e6f)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:36.828215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727d093-eb4d-48b5-8f8c-80420ca0f749 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:36.829565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 564/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:02:36 2022] GET /identity/v3/roles/b5e411fc96434df69c6cc1df951b3e6f => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:36.835363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a021d-6012-4f11-9abf-2b4dd52fbde7 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:36.867683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:36.931363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a021d-6012-4f11-9abf-2b4dd52fbde7 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:36.965001 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5ddca-a23a-48c9-8e54-ab65c1c8f11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:36.965735 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5ddca-a23a-48c9-8e54-ab65c1c8f11f None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:36.966019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5ddca-a23a-48c9-8e54-ab65c1c8f11f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:36.966293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5ddca-a23a-48c9-8e54-ab65c1c8f11f None admin] PATH_INFO: `/v3/users/bc2a1ea8dedb40aca1aa068f5fb0e575` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:36.978756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5ddca-a23a-48c9-8e54-ab65c1c8f11f None admin] RBAC: Authorizing `identity:get_user(user_id=bc2a1ea8dedb40aca1aa068f5fb0e575)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:36.980186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5ddca-a23a-48c9-8e54-ab65c1c8f11f None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:36.982455 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 550/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:02:36 2022] GET /identity/v3/users/bc2a1ea8dedb40aca1aa068f5fb0e575 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:36.988201 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727d093-eb4d-48b5-8f8c-80420ca0f749 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:37.085591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727d093-eb4d-48b5-8f8c-80420ca0f749 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:37.108531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:37.118782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee527e6-d665-4539-b2af-8c9545ce6988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:37.119375 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee527e6-d665-4539-b2af-8c9545ce6988 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:37.119654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee527e6-d665-4539-b2af-8c9545ce6988 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:37.119927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee527e6-d665-4539-b2af-8c9545ce6988 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:37.185499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee527e6-d665-4539-b2af-8c9545ce6988 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:37.186932 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee527e6-d665-4539-b2af-8c9545ce6988 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:37.282532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 565/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:02:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:02:37.288484 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5ddca-a23a-48c9-8e54-ab65c1c8f11f None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:37.384601 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:37.392885 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5ddca-a23a-48c9-8e54-ab65c1c8f11f None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:37.427164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9b444-5a14-4f8a-9879-cd502f0c8e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:37.427773 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9b444-5a14-4f8a-9879-cd502f0c8e4c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:37.428050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9b444-5a14-4f8a-9879-cd502f0c8e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:37.428318 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9b444-5a14-4f8a-9879-cd502f0c8e4c None admin] PATH_INFO: `/v3/projects/46f05f061b4542feb4e8f09d6fd141a6` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:37.429464 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9b444-5a14-4f8a-9879-cd502f0c8e4c None admin] RBAC: Authorizing `identity:get_project(project_id=46f05f061b4542feb4e8f09d6fd141a6)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:37.430880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9b444-5a14-4f8a-9879-cd502f0c8e4c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:37.432462 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 551/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:02:37 2022] GET /identity/v3/projects/46f05f061b4542feb4e8f09d6fd141a6 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:37.438989 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee527e6-d665-4539-b2af-8c9545ce6988 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:37.462647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:37.534727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee527e6-d665-4539-b2af-8c9545ce6988 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:37.567128 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220803c-d23b-4184-a5aa-4d8da0d342b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:37.567711 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220803c-d23b-4184-a5aa-4d8da0d342b3 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:37.567993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220803c-d23b-4184-a5aa-4d8da0d342b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:37.568265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220803c-d23b-4184-a5aa-4d8da0d342b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:37.569013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220803c-d23b-4184-a5aa-4d8da0d342b3 None admin] RBAC: Adding query filter params (role.id=b5e411fc96434df69c6cc1df951b3e6f, scope.project.id=46f05f061b4542feb4e8f09d6fd141a6, user.id=bc2a1ea8dedb40aca1aa068f5fb0e575) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:37.569322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220803c-d23b-4184-a5aa-4d8da0d342b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:37.570735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220803c-d23b-4184-a5aa-4d8da0d342b3 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:37.576484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 566/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:02:37 2022] GET /identity/v3/role_assignments?user.id=bc2a1ea8dedb40aca1aa068f5fb0e575&scope.project.id=46f05f061b4542feb4e8f09d6fd141a6&role.id=b5e411fc96434df69c6cc1df951b3e6f => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:39.443560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf33541-f49c-40a6-9b89-3c1c5ba6a288 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:39.444114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf33541-f49c-40a6-9b89-3c1c5ba6a288 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:39.444114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf33541-f49c-40a6-9b89-3c1c5ba6a288 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:39.445491 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 552/1118] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:39.450679 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03f051-2214-4f78-bb58-80118df53276 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:39.451061 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03f051-2214-4f78-bb58-80118df53276 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:39.451254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03f051-2214-4f78-bb58-80118df53276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:39.469826 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-3e03f051-2214-4f78-bb58-80118df53276 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:39.471019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-3e03f051-2214-4f78-bb58-80118df53276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:39.534613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 567/1119] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:39.798336 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf33541-f49c-40a6-9b89-3c1c5ba6a288 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:39.898855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf33541-f49c-40a6-9b89-3c1c5ba6a288 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:39.932340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812df3fc-823c-4025-a1e3-403a7f908118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:39.932984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812df3fc-823c-4025-a1e3-403a7f908118 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:39.933269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812df3fc-823c-4025-a1e3-403a7f908118 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:39.933544 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812df3fc-823c-4025-a1e3-403a7f908118 None admin] PATH_INFO: `/v3/roles/b5e411fc96434df69c6cc1df951b3e6f` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:39.935760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812df3fc-823c-4025-a1e3-403a7f908118 None admin] RBAC: Authorizing `identity:get_role(role_id=b5e411fc96434df69c6cc1df951b3e6f)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:39.937161 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812df3fc-823c-4025-a1e3-403a7f908118 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:39.938471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 553/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:02:39 2022] GET /identity/v3/roles/b5e411fc96434df69c6cc1df951b3e6f => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:39.944402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03f051-2214-4f78-bb58-80118df53276 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:39.962555 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:40.040506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03f051-2214-4f78-bb58-80118df53276 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:40.074048 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ee0ee-10f9-4417-b254-1cd2b3fe2d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:40.074845 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ee0ee-10f9-4417-b254-1cd2b3fe2d40 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:40.075132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ee0ee-10f9-4417-b254-1cd2b3fe2d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:40.075406 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ee0ee-10f9-4417-b254-1cd2b3fe2d40 None admin] PATH_INFO: `/v3/users/bc2a1ea8dedb40aca1aa068f5fb0e575` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:40.076517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ee0ee-10f9-4417-b254-1cd2b3fe2d40 None admin] RBAC: Authorizing `identity:get_user(user_id=bc2a1ea8dedb40aca1aa068f5fb0e575)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:40.077983 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ee0ee-10f9-4417-b254-1cd2b3fe2d40 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:40.079541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 568/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:02:39 2022] GET /identity/v3/users/bc2a1ea8dedb40aca1aa068f5fb0e575 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:40.085310 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812df3fc-823c-4025-a1e3-403a7f908118 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:40.112757 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:40.181016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812df3fc-823c-4025-a1e3-403a7f908118 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:40.213780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379907dd-8353-490b-ba96-412d32992b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:40.214390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379907dd-8353-490b-ba96-412d32992b4a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:40.214670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379907dd-8353-490b-ba96-412d32992b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:40.214938 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379907dd-8353-490b-ba96-412d32992b4a None admin] PATH_INFO: `/v3/projects/46f05f061b4542feb4e8f09d6fd141a6` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:40.216138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379907dd-8353-490b-ba96-412d32992b4a None admin] RBAC: Authorizing `identity:get_project(project_id=46f05f061b4542feb4e8f09d6fd141a6)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:40.217587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379907dd-8353-490b-ba96-412d32992b4a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:40.219181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 554/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:02:40 2022] GET /identity/v3/projects/46f05f061b4542feb4e8f09d6fd141a6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:40.225103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ee0ee-10f9-4417-b254-1cd2b3fe2d40 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:40.306716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:40.321025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ee0ee-10f9-4417-b254-1cd2b3fe2d40 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:40.354312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db06384-74b7-46dc-b5f2-39b3896c0997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:40.354884 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db06384-74b7-46dc-b5f2-39b3896c0997 None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:40.355162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db06384-74b7-46dc-b5f2-39b3896c0997 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:40.355433 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db06384-74b7-46dc-b5f2-39b3896c0997 None admin] PATH_INFO: `/v3/projects/46f05f061b4542feb4e8f09d6fd141a6/users/bc2a1ea8dedb40aca1aa068f5fb0e575/roles/b5e411fc96434df69c6cc1df951b3e6f` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:40.357668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db06384-74b7-46dc-b5f2-39b3896c0997 None admin] RBAC: Authorizing `identity:create_grant(project_id=46f05f061b4542feb4e8f09d6fd141a6, user_id=bc2a1ea8dedb40aca1aa068f5fb0e575, role_id=b5e411fc96434df69c6cc1df951b3e6f)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:40.359085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db06384-74b7-46dc-b5f2-39b3896c0997 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:40.463507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 569/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:02:40 2022] PUT /identity/v3/projects/46f05f061b4542feb4e8f09d6fd141a6/users/bc2a1ea8dedb40aca1aa068f5fb0e575/roles/b5e411fc96434df69c6cc1df951b3e6f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:02:42.323586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc5e9c-8acd-4f81-8c8f-6efef6aad468 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:42.324084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc5e9c-8acd-4f81-8c8f-6efef6aad468 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:42.324084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc5e9c-8acd-4f81-8c8f-6efef6aad468 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:42.325917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 555/1124] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:42.330964 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98655d31-245a-4fc8-af54-4d59e3c287f1 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:42.331350 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98655d31-245a-4fc8-af54-4d59e3c287f1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:42.331513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98655d31-245a-4fc8-af54-4d59e3c287f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:42.349644 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-98655d31-245a-4fc8-af54-4d59e3c287f1 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:42.350819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-98655d31-245a-4fc8-af54-4d59e3c287f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:42.414979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 570/1125] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:42.693094 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc5e9c-8acd-4f81-8c8f-6efef6aad468 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:42.785227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:42.791664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc5e9c-8acd-4f81-8c8f-6efef6aad468 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:42.827339 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9baaf-7851-4b50-80aa-ca981cb344a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:42.827951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9baaf-7851-4b50-80aa-ca981cb344a0 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:42.828230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9baaf-7851-4b50-80aa-ca981cb344a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:42.828504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9baaf-7851-4b50-80aa-ca981cb344a0 None admin] PATH_INFO: `/v3/roles/b5e411fc96434df69c6cc1df951b3e6f` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:42.829897 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9baaf-7851-4b50-80aa-ca981cb344a0 None admin] RBAC: Authorizing `identity:get_role(role_id=b5e411fc96434df69c6cc1df951b3e6f)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:42.831245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9baaf-7851-4b50-80aa-ca981cb344a0 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:42.832552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 556/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:02:42 2022] GET /identity/v3/roles/b5e411fc96434df69c6cc1df951b3e6f => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:42.838287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98655d31-245a-4fc8-af54-4d59e3c287f1 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:42.911393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:42.933028 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98655d31-245a-4fc8-af54-4d59e3c287f1 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:42.970004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e464e-1547-49a3-8240-1a99179b2220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:42.970549 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e464e-1547-49a3-8240-1a99179b2220 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:42.970828 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e464e-1547-49a3-8240-1a99179b2220 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:42.971096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e464e-1547-49a3-8240-1a99179b2220 None admin] PATH_INFO: `/v3/users/bc2a1ea8dedb40aca1aa068f5fb0e575` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:42.972193 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e464e-1547-49a3-8240-1a99179b2220 None admin] RBAC: Authorizing `identity:get_user(user_id=bc2a1ea8dedb40aca1aa068f5fb0e575)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:42.973639 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e464e-1547-49a3-8240-1a99179b2220 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:42.975187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 571/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:02:42 2022] GET /identity/v3/users/bc2a1ea8dedb40aca1aa068f5fb0e575 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:42.981042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9baaf-7851-4b50-80aa-ca981cb344a0 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:43.077210 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9baaf-7851-4b50-80aa-ca981cb344a0 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:43.110112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f267f3b-4792-47b1-977d-218d2aee6ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:43.110693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f267f3b-4792-47b1-977d-218d2aee6ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:43.110973 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f267f3b-4792-47b1-977d-218d2aee6ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:43.111245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f267f3b-4792-47b1-977d-218d2aee6ce9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:43.141157 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:43.177206 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f267f3b-4792-47b1-977d-218d2aee6ce9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:43.178637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f267f3b-4792-47b1-977d-218d2aee6ce9 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:43.271964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 557/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:02:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:02:43.277947 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e464e-1547-49a3-8240-1a99179b2220 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:43.374088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e464e-1547-49a3-8240-1a99179b2220 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:43.406946 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00202958-4a2a-4aa8-95c8-07e7d9aadd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:43.407560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00202958-4a2a-4aa8-95c8-07e7d9aadd5a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:43.407843 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00202958-4a2a-4aa8-95c8-07e7d9aadd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:43.408115 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00202958-4a2a-4aa8-95c8-07e7d9aadd5a None admin] PATH_INFO: `/v3/projects/46f05f061b4542feb4e8f09d6fd141a6` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:43.409239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00202958-4a2a-4aa8-95c8-07e7d9aadd5a None admin] RBAC: Authorizing `identity:get_project(project_id=46f05f061b4542feb4e8f09d6fd141a6)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:43.410662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00202958-4a2a-4aa8-95c8-07e7d9aadd5a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:43.412265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 572/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:02:43 2022] GET /identity/v3/projects/46f05f061b4542feb4e8f09d6fd141a6 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:43.419117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f267f3b-4792-47b1-977d-218d2aee6ce9 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:43.478477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:43.514781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f267f3b-4792-47b1-977d-218d2aee6ce9 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:43.547588 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a197aff-2bd0-47de-b3f9-b74eb44a4bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:43.548107 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a197aff-2bd0-47de-b3f9-b74eb44a4bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:43.548382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a197aff-2bd0-47de-b3f9-b74eb44a4bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:43.548681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a197aff-2bd0-47de-b3f9-b74eb44a4bf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:43.549379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a197aff-2bd0-47de-b3f9-b74eb44a4bf8 None admin] RBAC: Adding query filter params (role.id=b5e411fc96434df69c6cc1df951b3e6f, scope.project.id=46f05f061b4542feb4e8f09d6fd141a6, user.id=bc2a1ea8dedb40aca1aa068f5fb0e575) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:43.549682 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a197aff-2bd0-47de-b3f9-b74eb44a4bf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:43.551343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a197aff-2bd0-47de-b3f9-b74eb44a4bf8 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:43.556892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 558/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:02:43 2022] GET /identity/v3/role_assignments?user.id=bc2a1ea8dedb40aca1aa068f5fb0e575&scope.project.id=46f05f061b4542feb4e8f09d6fd141a6&role.id=b5e411fc96434df69c6cc1df951b3e6f => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:45.454136 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bb932-da4a-49cf-8362-472e9a6e3804 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:45.454692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bb932-da4a-49cf-8362-472e9a6e3804 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:45.454692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bb932-da4a-49cf-8362-472e9a6e3804 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:45.456018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 573/1131] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:45.461402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ccc8e-2301-4f09-9919-c12dfddebd8c None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:45.461809 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ccc8e-2301-4f09-9919-c12dfddebd8c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:45.461949 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ccc8e-2301-4f09-9919-c12dfddebd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:45.479824 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-892ccc8e-2301-4f09-9919-c12dfddebd8c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:45.481626 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-892ccc8e-2301-4f09-9919-c12dfddebd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:45.544164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 559/1132] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:45.811437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bb932-da4a-49cf-8362-472e9a6e3804 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:45.826012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:45.914725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bb932-da4a-49cf-8362-472e9a6e3804 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:45.951520 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4366c-ed38-48e0-933c-c8169ba43c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:45.952136 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4366c-ed38-48e0-933c-c8169ba43c5a None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:45.952420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4366c-ed38-48e0-933c-c8169ba43c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:45.952722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4366c-ed38-48e0-933c-c8169ba43c5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:45.953837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4366c-ed38-48e0-933c-c8169ba43c5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:45.955273 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4366c-ed38-48e0-933c-c8169ba43c5a None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:45.956854 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 574/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:45.962816 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ccc8e-2301-4f09-9919-c12dfddebd8c None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:46.018078 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:46.058962 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ccc8e-2301-4f09-9919-c12dfddebd8c None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:46.091605 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7641a35-27a6-4924-8401-73b719760774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:46.092133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7641a35-27a6-4924-8401-73b719760774 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:46.092412 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7641a35-27a6-4924-8401-73b719760774 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:46.092703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7641a35-27a6-4924-8401-73b719760774 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:46.093346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7641a35-27a6-4924-8401-73b719760774 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:46.094749 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7641a35-27a6-4924-8401-73b719760774 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:46.189892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 560/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 12 12:02:45 2022] POST /identity/v3/projects => generated 311 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:02:48.095216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea7606-50d3-4b04-a4d1-af419710eefd None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:48.095743 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea7606-50d3-4b04-a4d1-af419710eefd None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:48.095743 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea7606-50d3-4b04-a4d1-af419710eefd None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:48.097049 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 575/1135] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:48.102065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc634d-1b6f-4861-bb72-b6ac0267a854 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:48.102446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc634d-1b6f-4861-bb72-b6ac0267a854 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:48.102652 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc634d-1b6f-4861-bb72-b6ac0267a854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:48.121304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-3afc634d-1b6f-4861-bb72-b6ac0267a854 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:48.122623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-3afc634d-1b6f-4861-bb72-b6ac0267a854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:48.186117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 561/1136] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:48.454973 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea7606-50d3-4b04-a4d1-af419710eefd None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:48.534112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:48.554110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea7606-50d3-4b04-a4d1-af419710eefd None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:48.587419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afb325-5515-4fad-bce3-e5de427c81e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:48.588021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afb325-5515-4fad-bce3-e5de427c81e4 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:48.588301 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afb325-5515-4fad-bce3-e5de427c81e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:48.588573 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afb325-5515-4fad-bce3-e5de427c81e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:48.590203 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afb325-5515-4fad-bce3-e5de427c81e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:48.591617 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afb325-5515-4fad-bce3-e5de427c81e4 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:48.593173 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 576/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:02:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:48.598925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc634d-1b6f-4861-bb72-b6ac0267a854 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:48.638540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:48.694257 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc634d-1b6f-4861-bb72-b6ac0267a854 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:48.727663 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f952046-6b67-451f-9b10-ad3f1305e07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:48.728195 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f952046-6b67-451f-9b10-ad3f1305e07f None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:48.728469 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f952046-6b67-451f-9b10-ad3f1305e07f None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:48.728780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f952046-6b67-451f-9b10-ad3f1305e07f None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:48.729419 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f952046-6b67-451f-9b10-ad3f1305e07f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:48.731524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f952046-6b67-451f-9b10-ad3f1305e07f None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:48.806162 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 562/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:02:48 2022] POST /identity/v3/users => generated 286 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:02:50.720804 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc930d-eb28-43f1-bb2b-4967b68213cf None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:50.721380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc930d-eb28-43f1-bb2b-4967b68213cf None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:50.721380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc930d-eb28-43f1-bb2b-4967b68213cf None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:50.722719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 577/1139] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:50.727747 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4450b2c-df13-4064-877f-268f21f0a9f2 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:50.728192 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4450b2c-df13-4064-877f-268f21f0a9f2 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:50.728299 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4450b2c-df13-4064-877f-268f21f0a9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:50.746581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-e4450b2c-df13-4064-877f-268f21f0a9f2 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:50.747756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-e4450b2c-df13-4064-877f-268f21f0a9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:50.812357 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 563/1140] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:02:51.082544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc930d-eb28-43f1-bb2b-4967b68213cf None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:51.180732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc930d-eb28-43f1-bb2b-4967b68213cf None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:51.212935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d59696-8510-4d48-b6f6-513342a03fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:51.213540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d59696-8510-4d48-b6f6-513342a03fae None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:51.213817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d59696-8510-4d48-b6f6-513342a03fae None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:51.214133 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d59696-8510-4d48-b6f6-513342a03fae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:51.224127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d59696-8510-4d48-b6f6-513342a03fae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-48d59696-8510-4d48-b6f6-513342a03fae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:02:51.228856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:51.230434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:51.231430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:51.231430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:51.231430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:51.231430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:51.231430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:02:51.231430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:02:51.231430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:02:51.231430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:02:51.231430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:02:51.231430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:02:51.231430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:02:51.231430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:51.231430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 578/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:02:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:51.236503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4450b2c-df13-4064-877f-268f21f0a9f2 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:51.263508 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:51.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4450b2c-df13-4064-877f-268f21f0a9f2 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:51.367741 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb6adc-f50d-4125-8872-d6a9fb70fff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:51.368328 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb6adc-f50d-4125-8872-d6a9fb70fff1 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:51.368608 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb6adc-f50d-4125-8872-d6a9fb70fff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:51.368921 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb6adc-f50d-4125-8872-d6a9fb70fff1 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:51.369576 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb6adc-f50d-4125-8872-d6a9fb70fff1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:51.369878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb6adc-f50d-4125-8872-d6a9fb70fff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:51.371935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb6adc-f50d-4125-8872-d6a9fb70fff1 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:51.379599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 564/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:02:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:51.385506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d59696-8510-4d48-b6f6-513342a03fae None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:51.396839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:51.485365 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d59696-8510-4d48-b6f6-513342a03fae None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:51.517910 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02598685-e007-4e68-aa7e-fda29e5cd85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:51.518528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02598685-e007-4e68-aa7e-fda29e5cd85e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:51.518814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02598685-e007-4e68-aa7e-fda29e5cd85e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:51.519090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02598685-e007-4e68-aa7e-fda29e5cd85e None admin] PATH_INFO: `/v3/users/ff94c278ed9c4511b1a37bacfdb39b17` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:51.532496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02598685-e007-4e68-aa7e-fda29e5cd85e None admin] RBAC: Authorizing `identity:get_user(user_id=ff94c278ed9c4511b1a37bacfdb39b17)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:51.533963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02598685-e007-4e68-aa7e-fda29e5cd85e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:51.535550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 579/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:02:51 2022] GET /identity/v3/users/ff94c278ed9c4511b1a37bacfdb39b17 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:51.541433 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb6adc-f50d-4125-8872-d6a9fb70fff1 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:51.628367 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:51.640207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb6adc-f50d-4125-8872-d6a9fb70fff1 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:51.673627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bee18-0af9-4b4b-8f8f-aa443c9158ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:51.674213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bee18-0af9-4b4b-8f8f-aa443c9158ce None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:51.674498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bee18-0af9-4b4b-8f8f-aa443c9158ce None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:51.674771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bee18-0af9-4b4b-8f8f-aa443c9158ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:51.741686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7bee18-0af9-4b4b-8f8f-aa443c9158ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:51.743118 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7bee18-0af9-4b4b-8f8f-aa443c9158ce None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:51.827808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:51.841016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 565/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:02:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:02:51.847008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02598685-e007-4e68-aa7e-fda29e5cd85e None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:51.910018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:51.942879 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02598685-e007-4e68-aa7e-fda29e5cd85e None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:51.975977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134f46a-c997-4135-ae0f-8ea81223e078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:51.976590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134f46a-c997-4135-ae0f-8ea81223e078 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:51.976904 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134f46a-c997-4135-ae0f-8ea81223e078 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:51.977180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134f46a-c997-4135-ae0f-8ea81223e078 None admin] PATH_INFO: `/v3/projects/e1db602e904a4571a50e98e6f9ad7da2` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:51.978279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134f46a-c997-4135-ae0f-8ea81223e078 None admin] RBAC: Authorizing `identity:get_project(project_id=e1db602e904a4571a50e98e6f9ad7da2)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:51.979818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134f46a-c997-4135-ae0f-8ea81223e078 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:51.981475 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 580/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:02:51 2022] GET /identity/v3/projects/e1db602e904a4571a50e98e6f9ad7da2 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:51.987590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bee18-0af9-4b4b-8f8f-aa443c9158ce None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:52.082525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bee18-0af9-4b4b-8f8f-aa443c9158ce None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:52.115043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022dd8c-e040-4c7c-bbd1-41893a8f2f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:52.115626 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022dd8c-e040-4c7c-bbd1-41893a8f2f1a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:52.115910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022dd8c-e040-4c7c-bbd1-41893a8f2f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:52.116187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022dd8c-e040-4c7c-bbd1-41893a8f2f1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:52.116932 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022dd8c-e040-4c7c-bbd1-41893a8f2f1a None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=e1db602e904a4571a50e98e6f9ad7da2, user.id=ff94c278ed9c4511b1a37bacfdb39b17) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:52.117240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022dd8c-e040-4c7c-bbd1-41893a8f2f1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:52.118648 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022dd8c-e040-4c7c-bbd1-41893a8f2f1a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:52.124495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 566/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:02:51 2022] GET /identity/v3/role_assignments?user.id=ff94c278ed9c4511b1a37bacfdb39b17&scope.project.id=e1db602e904a4571a50e98e6f9ad7da2&role.id=906e3a8a6c7540a38f629f2567133347 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:53.981674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dbfb0-ffbd-4755-ad00-6a7523cfd28c None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:53.982188 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dbfb0-ffbd-4755-ad00-6a7523cfd28c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:53.982304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437dbfb0-ffbd-4755-ad00-6a7523cfd28c None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:53.983554 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 581/1147] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:53.988772 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61accf8-c20d-4cec-b6a1-8afb74b2efbe None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:53.989179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61accf8-c20d-4cec-b6a1-8afb74b2efbe None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:53.989321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61accf8-c20d-4cec-b6a1-8afb74b2efbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:54.006925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-e61accf8-c20d-4cec-b6a1-8afb74b2efbe None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:54.008099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-e61accf8-c20d-4cec-b6a1-8afb74b2efbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:54.053908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:54.074138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 567/1148] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:54.343971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437dbfb0-ffbd-4755-ad00-6a7523cfd28c None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:54.442584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437dbfb0-ffbd-4755-ad00-6a7523cfd28c None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:54.474850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3cec6-eb92-4a62-b432-d793b4934d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:54.475457 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3cec6-eb92-4a62-b432-d793b4934d89 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:54.475735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3cec6-eb92-4a62-b432-d793b4934d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:54.476001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3cec6-eb92-4a62-b432-d793b4934d89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:54.477799 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:54.486855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3cec6-eb92-4a62-b432-d793b4934d89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-89d3cec6-eb92-4a62-b432-d793b4934d89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:02:54.491465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:54.492531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:54.493425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:54.493425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:54.493425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:54.493425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:54.493425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:02:54.493425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:02:54.493425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:02:54.493425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:02:54.493425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:02:54.493425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:02:54.493425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:02:54.493425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:02:54.493425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 582/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:02:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:54.498712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61accf8-c20d-4cec-b6a1-8afb74b2efbe None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:54.593751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61accf8-c20d-4cec-b6a1-8afb74b2efbe None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:54.627392 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f94e8-e829-426a-b7e7-337838edacc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:54.627919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f94e8-e829-426a-b7e7-337838edacc3 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:54.628196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f94e8-e829-426a-b7e7-337838edacc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:54.628488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f94e8-e829-426a-b7e7-337838edacc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:54.629146 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f94e8-e829-426a-b7e7-337838edacc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:54.629447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f94e8-e829-426a-b7e7-337838edacc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:54.630788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f94e8-e829-426a-b7e7-337838edacc3 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:54.638487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 568/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:02:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:54.644195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3cec6-eb92-4a62-b432-d793b4934d89 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:54.741264 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3cec6-eb92-4a62-b432-d793b4934d89 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:54.773747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83680ed-2990-496a-82e5-111189d4ac1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:54.774351 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83680ed-2990-496a-82e5-111189d4ac1e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:54.774635 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83680ed-2990-496a-82e5-111189d4ac1e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:54.774912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83680ed-2990-496a-82e5-111189d4ac1e None admin] PATH_INFO: `/v3/users/ff94c278ed9c4511b1a37bacfdb39b17` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:54.776147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83680ed-2990-496a-82e5-111189d4ac1e None admin] RBAC: Authorizing `identity:get_user(user_id=ff94c278ed9c4511b1a37bacfdb39b17)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:54.777629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83680ed-2990-496a-82e5-111189d4ac1e None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:54.779171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 583/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:02:54 2022] GET /identity/v3/users/ff94c278ed9c4511b1a37bacfdb39b17 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:54.784944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f94e8-e829-426a-b7e7-337838edacc3 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:54.830872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:54.883821 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f94e8-e829-426a-b7e7-337838edacc3 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:54.915818 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd361c17-97eb-4a83-a665-44e970903b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:54.916433 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd361c17-97eb-4a83-a665-44e970903b72 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:54.917287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd361c17-97eb-4a83-a665-44e970903b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:54.917700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd361c17-97eb-4a83-a665-44e970903b72 None admin] PATH_INFO: `/v3/projects/e1db602e904a4571a50e98e6f9ad7da2` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:54.918819 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd361c17-97eb-4a83-a665-44e970903b72 None admin] RBAC: Authorizing `identity:get_project(project_id=e1db602e904a4571a50e98e6f9ad7da2)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:54.920250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd361c17-97eb-4a83-a665-44e970903b72 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:54.921898 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 569/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:02:54 2022] GET /identity/v3/projects/e1db602e904a4571a50e98e6f9ad7da2 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:54.928130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83680ed-2990-496a-82e5-111189d4ac1e None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:54.988377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:55.025698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83680ed-2990-496a-82e5-111189d4ac1e None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:55.059227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19cc93-2fc9-4321-8d3b-d1147c41d145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:55.059860 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19cc93-2fc9-4321-8d3b-d1147c41d145 None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:55.060139 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19cc93-2fc9-4321-8d3b-d1147c41d145 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:55.060410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19cc93-2fc9-4321-8d3b-d1147c41d145 None admin] PATH_INFO: `/v3/projects/e1db602e904a4571a50e98e6f9ad7da2/users/ff94c278ed9c4511b1a37bacfdb39b17/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:55.062279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19cc93-2fc9-4321-8d3b-d1147c41d145 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1db602e904a4571a50e98e6f9ad7da2, user_id=ff94c278ed9c4511b1a37bacfdb39b17, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:55.063695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19cc93-2fc9-4321-8d3b-d1147c41d145 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:55.164655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 584/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:02:54 2022] PUT /identity/v3/projects/e1db602e904a4571a50e98e6f9ad7da2/users/ff94c278ed9c4511b1a37bacfdb39b17/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:02:57.024716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9badc-2f10-466e-b8b5-7bd1ccd15491 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:57.025234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9badc-2f10-466e-b8b5-7bd1ccd15491 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:57.025323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9badc-2f10-466e-b8b5-7bd1ccd15491 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:57.027196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 570/1154] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:02:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:02:57.032377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac49c0f-3874-4d57-b8d8-24193db1144c None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:57.032821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac49c0f-3874-4d57-b8d8-24193db1144c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:57.032956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac49c0f-3874-4d57-b8d8-24193db1144c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:57.050634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-0ac49c0f-3874-4d57-b8d8-24193db1144c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:02:57.051801 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0ac49c0f-3874-4d57-b8d8-24193db1144c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:02:57.115534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 585/1155] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:02:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:02:57.430093 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9badc-2f10-466e-b8b5-7bd1ccd15491 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:57.536162 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9badc-2f10-466e-b8b5-7bd1ccd15491 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:57.549892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:57.577443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e40e18-d50b-449e-b7bf-9c948c0f7476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:57.578061 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e40e18-d50b-449e-b7bf-9c948c0f7476 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:57.578343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e40e18-d50b-449e-b7bf-9c948c0f7476 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:57.578613 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e40e18-d50b-449e-b7bf-9c948c0f7476 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:57.590336 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e40e18-d50b-449e-b7bf-9c948c0f7476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-54e40e18-d50b-449e-b7bf-9c948c0f7476 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:02:57.594948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:02:57.596117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:02:57.597232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:02:57.597232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:02:57.597232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:02:57.597232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:02:57.597232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:02:57.597232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:02:57.597232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:02:57.597232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:02:57.597232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:02:57.597232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:02:57.597232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:02:57.597232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:02:57.597232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 571/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:02:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:02:57.602330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac49c0f-3874-4d57-b8d8-24193db1144c None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:57.662724 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:57.710495 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac49c0f-3874-4d57-b8d8-24193db1144c None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:57.744165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47a354-4397-4266-a705-bf4fea23930f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:57.745680 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47a354-4397-4266-a705-bf4fea23930f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:57.745959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47a354-4397-4266-a705-bf4fea23930f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:57.746233 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47a354-4397-4266-a705-bf4fea23930f None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:57.746897 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47a354-4397-4266-a705-bf4fea23930f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:57.747201 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47a354-4397-4266-a705-bf4fea23930f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:57.748573 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47a354-4397-4266-a705-bf4fea23930f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:57.756850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 586/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:02:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:57.762981 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e40e18-d50b-449e-b7bf-9c948c0f7476 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:57.877511 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e40e18-d50b-449e-b7bf-9c948c0f7476 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:57.915771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad93d4-8529-414a-adc9-98ca9281aa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:57.917293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad93d4-8529-414a-adc9-98ca9281aa3b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:57.917755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad93d4-8529-414a-adc9-98ca9281aa3b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:57.918204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad93d4-8529-414a-adc9-98ca9281aa3b None admin] PATH_INFO: `/v3/users/ff94c278ed9c4511b1a37bacfdb39b17` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:57.919986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad93d4-8529-414a-adc9-98ca9281aa3b None admin] RBAC: Authorizing `identity:get_user(user_id=ff94c278ed9c4511b1a37bacfdb39b17)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:57.922477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad93d4-8529-414a-adc9-98ca9281aa3b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:57.924454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 572/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:02:57 2022] GET /identity/v3/users/ff94c278ed9c4511b1a37bacfdb39b17 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:57.930758 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47a354-4397-4266-a705-bf4fea23930f None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:58.033449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47a354-4397-4266-a705-bf4fea23930f None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:58.046844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:58.066405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19382b-cae1-449a-afd1-9356e7557109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:58.067000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19382b-cae1-449a-afd1-9356e7557109 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:58.067276 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19382b-cae1-449a-afd1-9356e7557109 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:58.067543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19382b-cae1-449a-afd1-9356e7557109 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:58.131388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19382b-cae1-449a-afd1-9356e7557109 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:58.132840 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19382b-cae1-449a-afd1-9356e7557109 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:58.226519 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 587/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:02:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:02:58.232480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad93d4-8529-414a-adc9-98ca9281aa3b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:58.274318 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:58.331204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad93d4-8529-414a-adc9-98ca9281aa3b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:58.362760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199aeb1-7c2f-48d5-8e20-e9c1611d0059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:58.363375 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199aeb1-7c2f-48d5-8e20-e9c1611d0059 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:58.363654 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199aeb1-7c2f-48d5-8e20-e9c1611d0059 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:58.363928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199aeb1-7c2f-48d5-8e20-e9c1611d0059 None admin] PATH_INFO: `/v3/projects/e1db602e904a4571a50e98e6f9ad7da2` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:58.365053 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199aeb1-7c2f-48d5-8e20-e9c1611d0059 None admin] RBAC: Authorizing `identity:get_project(project_id=e1db602e904a4571a50e98e6f9ad7da2)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:58.366472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199aeb1-7c2f-48d5-8e20-e9c1611d0059 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:58.368077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 573/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:02:58 2022] GET /identity/v3/projects/e1db602e904a4571a50e98e6f9ad7da2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:02:58.374743 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19382b-cae1-449a-afd1-9356e7557109 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:02:58.389748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:02:58.470143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19382b-cae1-449a-afd1-9356e7557109 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:02:58.502566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c0ef3-2322-40a6-999e-1790256c4cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:02:58.503178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c0ef3-2322-40a6-999e-1790256c4cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:02:58.503457 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c0ef3-2322-40a6-999e-1790256c4cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:02:58.503728 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c0ef3-2322-40a6-999e-1790256c4cd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:02:58.504445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c0ef3-2322-40a6-999e-1790256c4cd9 None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=e1db602e904a4571a50e98e6f9ad7da2, user.id=ff94c278ed9c4511b1a37bacfdb39b17) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:02:58.504787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c0ef3-2322-40a6-999e-1790256c4cd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:02:58.506196 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c0ef3-2322-40a6-999e-1790256c4cd9 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:02:58.512333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 588/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:02:58 2022] GET /identity/v3/role_assignments?user.id=ff94c278ed9c4511b1a37bacfdb39b17&scope.project.id=e1db602e904a4571a50e98e6f9ad7da2&role.id=906e3a8a6c7540a38f629f2567133347 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:03:00.422708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0196089-fc4b-4ac8-8ee0-de4779a3db3c None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:00.423267 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0196089-fc4b-4ac8-8ee0-de4779a3db3c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:00.423267 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0196089-fc4b-4ac8-8ee0-de4779a3db3c None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:00.424575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 574/1162] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:03:00.430036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974b741-c969-4654-89bb-4b433b26393b None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:00.430407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974b741-c969-4654-89bb-4b433b26393b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:00.430588 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f974b741-c969-4654-89bb-4b433b26393b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:00.448248 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f974b741-c969-4654-89bb-4b433b26393b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:03:00.449487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f974b741-c969-4654-89bb-4b433b26393b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:03:00.513408 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 589/1163] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:03:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:03:00.783780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0196089-fc4b-4ac8-8ee0-de4779a3db3c None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:00.883870 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0196089-fc4b-4ac8-8ee0-de4779a3db3c None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:00.892266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:00.915154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0af0a-b668-482f-a2d4-20fab85178c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:00.915740 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0af0a-b668-482f-a2d4-20fab85178c5 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:00.916557 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0af0a-b668-482f-a2d4-20fab85178c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:00.916906 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0af0a-b668-482f-a2d4-20fab85178c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:00.984404 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0af0a-b668-482f-a2d4-20fab85178c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:00.985858 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0af0a-b668-482f-a2d4-20fab85178c5 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:01.079349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 575/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:03:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:03:01.085260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974b741-c969-4654-89bb-4b433b26393b None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:01.095150 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:01.182415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f974b741-c969-4654-89bb-4b433b26393b None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:01.214729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e0330-cc1a-41fd-bd7a-d4ff7c8ef56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:01.215348 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e0330-cc1a-41fd-bd7a-d4ff7c8ef56d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:01.215626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e0330-cc1a-41fd-bd7a-d4ff7c8ef56d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:01.215898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e0330-cc1a-41fd-bd7a-d4ff7c8ef56d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:01.222300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e0330-cc1a-41fd-bd7a-d4ff7c8ef56d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:01.223729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e0330-cc1a-41fd-bd7a-d4ff7c8ef56d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-a06e0330-cc1a-41fd-bd7a-d4ff7c8ef56d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:03:01.235302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:03:01.236439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:03:01.237438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:03:01.237438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:03:01.237438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:03:01.237438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:03:01.237438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:03:01.237438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:03:01.237438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:03:01.237438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:03:01.237438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:03:01.237438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:03:01.237438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 12 12:03:01.237438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:03:01.237438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 12 12:03:01.237438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:03:01.237438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 590/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 12 12:03:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:03:01.242318 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0af0a-b668-482f-a2d4-20fab85178c5 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:01.252298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:01.337683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0af0a-b668-482f-a2d4-20fab85178c5 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:01.370963 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab58531-ef6d-4a11-80aa-a153afab156a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:01.371548 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab58531-ef6d-4a11-80aa-a153afab156a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:01.371826 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab58531-ef6d-4a11-80aa-a153afab156a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:01.372097 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab58531-ef6d-4a11-80aa-a153afab156a None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:01.372749 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab58531-ef6d-4a11-80aa-a153afab156a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:03:01.373050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab58531-ef6d-4a11-80aa-a153afab156a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:01.374394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab58531-ef6d-4a11-80aa-a153afab156a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:01.381224 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 576/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Aug 12 12:03:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:03:03.240304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05339c-c815-44b5-b50b-0ddf1521dff8 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:03.240868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05339c-c815-44b5-b50b-0ddf1521dff8 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:03.240868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05339c-c815-44b5-b50b-0ddf1521dff8 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:03.242262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 591/1167] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:03:03.248021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a42d9b-ff1d-479f-8a62-50e5da65b610 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:03.248439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a42d9b-ff1d-479f-8a62-50e5da65b610 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:03.248569 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a42d9b-ff1d-479f-8a62-50e5da65b610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:03.266153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-52a42d9b-ff1d-479f-8a62-50e5da65b610 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:03:03.267321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-52a42d9b-ff1d-479f-8a62-50e5da65b610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:03:03.331802 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 577/1168] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:03:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:03:03.594711 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05339c-c815-44b5-b50b-0ddf1521dff8 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:03.660769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:03.694936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05339c-c815-44b5-b50b-0ddf1521dff8 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:03.727473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d7703-ad29-4c47-9225-178e49f72389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:03.728062 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d7703-ad29-4c47-9225-178e49f72389 None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:03.728976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d7703-ad29-4c47-9225-178e49f72389 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:03.729251 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d7703-ad29-4c47-9225-178e49f72389 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:03.729909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d7703-ad29-4c47-9225-178e49f72389 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:03.731271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d7703-ad29-4c47-9225-178e49f72389 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:03.806448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 592/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Fri Aug 12 12:03:03 2022] POST /identity/v3/domains => generated 267 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:03:05.715466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186903d-2392-40fd-82e3-852845417831 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:05.716095 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186903d-2392-40fd-82e3-852845417831 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:05.716095 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186903d-2392-40fd-82e3-852845417831 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:05.717386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 578/1170] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:03:05.722441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8ea4d-1ac0-41b4-8fb7-6fc82f6d5195 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:05.722853 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8ea4d-1ac0-41b4-8fb7-6fc82f6d5195 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:05.722990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c8ea4d-1ac0-41b4-8fb7-6fc82f6d5195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:05.741223 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-b7c8ea4d-1ac0-41b4-8fb7-6fc82f6d5195 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:03:05.742409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b7c8ea4d-1ac0-41b4-8fb7-6fc82f6d5195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:03:05.806920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 593/1171] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:03:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:03:06.078451 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186903d-2392-40fd-82e3-852845417831 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:06.081673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:06.182714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186903d-2392-40fd-82e3-852845417831 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:06.216099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a27ed0-ae9d-4cbb-99fb-8d7a4d51fd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:06.216731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a27ed0-ae9d-4cbb-99fb-8d7a4d51fd78 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:06.217023 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a27ed0-ae9d-4cbb-99fb-8d7a4d51fd78 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:06.217298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a27ed0-ae9d-4cbb-99fb-8d7a4d51fd78 None admin] PATH_INFO: `/v3/domains/8113929c5b8a4bcc86ec4aea096ea35b` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:06.227403 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a27ed0-ae9d-4cbb-99fb-8d7a4d51fd78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8113929c5b8a4bcc86ec4aea096ea35b)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:06.229704 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a27ed0-ae9d-4cbb-99fb-8d7a4d51fd78 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:06.231269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 579/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Fri Aug 12 12:03:06 2022] GET /identity/v3/domains/8113929c5b8a4bcc86ec4aea096ea35b => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:03:06.237294 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c8ea4d-1ac0-41b4-8fb7-6fc82f6d5195 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:06.293574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:06.333647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c8ea4d-1ac0-41b4-8fb7-6fc82f6d5195 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:06.366387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a06a8-4306-46ee-a7c9-39fb1b486a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:06.366974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a06a8-4306-46ee-a7c9-39fb1b486a95 None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:06.367253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a06a8-4306-46ee-a7c9-39fb1b486a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:06.367521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a06a8-4306-46ee-a7c9-39fb1b486a95 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:06.368163 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57a06a8-4306-46ee-a7c9-39fb1b486a95 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:06.370407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57a06a8-4306-46ee-a7c9-39fb1b486a95 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:06.432370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 594/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 12 12:03:06 2022] POST /identity/v3/projects => generated 361 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:03:08.338381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73ab92-0ee6-4e08-97d4-e6e0ee865919 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:08.338931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73ab92-0ee6-4e08-97d4-e6e0ee865919 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:08.338931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73ab92-0ee6-4e08-97d4-e6e0ee865919 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:08.340273 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 580/1174] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:03:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:03:08.345417 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918502a-a346-41cc-9040-5cf4361f6227 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:08.345819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918502a-a346-41cc-9040-5cf4361f6227 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:08.345976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918502a-a346-41cc-9040-5cf4361f6227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:08.363861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-7918502a-a346-41cc-9040-5cf4361f6227 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:03:08.365091 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7918502a-a346-41cc-9040-5cf4361f6227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:03:08.430404 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 595/1175] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:03:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:03:08.705436 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73ab92-0ee6-4e08-97d4-e6e0ee865919 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:08.766936 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:08.805485 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73ab92-0ee6-4e08-97d4-e6e0ee865919 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:08.838280 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88d5e7-0389-4248-bfdb-d725155e7be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:08.838893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88d5e7-0389-4248-bfdb-d725155e7be1 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:08.839174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88d5e7-0389-4248-bfdb-d725155e7be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:08.839443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88d5e7-0389-4248-bfdb-d725155e7be1 None admin] PATH_INFO: `/v3/domains/8113929c5b8a4bcc86ec4aea096ea35b` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:08.840544 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88d5e7-0389-4248-bfdb-d725155e7be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8113929c5b8a4bcc86ec4aea096ea35b)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:08.842037 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88d5e7-0389-4248-bfdb-d725155e7be1 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:08.843592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 581/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Fri Aug 12 12:03:08 2022] GET /identity/v3/domains/8113929c5b8a4bcc86ec4aea096ea35b => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:03:08.849384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918502a-a346-41cc-9040-5cf4361f6227 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:08.889296 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:08.946157 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918502a-a346-41cc-9040-5cf4361f6227 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:08.980558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bca6b-975a-4c60-962a-fae195a6599c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:08.981118 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bca6b-975a-4c60-962a-fae195a6599c None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:08.981503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bca6b-975a-4c60-962a-fae195a6599c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:08.981794 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bca6b-975a-4c60-962a-fae195a6599c None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:08.982448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bca6b-975a-4c60-962a-fae195a6599c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:08.983882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bca6b-975a-4c60-962a-fae195a6599c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:09.077919 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 596/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:03:08 2022] POST /identity/v3/users => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:03:11.007961 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d71e1-5bbf-4043-9069-0504c1e5ba47 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:11.008508 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d71e1-5bbf-4043-9069-0504c1e5ba47 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:11.008508 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d71e1-5bbf-4043-9069-0504c1e5ba47 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:11.010057 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 582/1178] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:03:11.015213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7160423-4dd0-47a9-957f-8f8711add6b6 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:11.015615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7160423-4dd0-47a9-957f-8f8711add6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:11.015766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7160423-4dd0-47a9-957f-8f8711add6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:11.034418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-b7160423-4dd0-47a9-957f-8f8711add6b6 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:03:11.035602 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b7160423-4dd0-47a9-957f-8f8711add6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:03:11.102013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 597/1179] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:03:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:03:11.369538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d71e1-5bbf-4043-9069-0504c1e5ba47 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:11.469149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d71e1-5bbf-4043-9069-0504c1e5ba47 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:11.495363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:11.500850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41f5bc-1b49-497f-b21a-63e1b5b910ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:11.505497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41f5bc-1b49-497f-b21a-63e1b5b910ee None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:11.505497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41f5bc-1b49-497f-b21a-63e1b5b910ee None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:11.505497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41f5bc-1b49-497f-b21a-63e1b5b910ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:11.512965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41f5bc-1b49-497f-b21a-63e1b5b910ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-6a41f5bc-1b49-497f-b21a-63e1b5b910ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:03:11.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:03:11.518755 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:03:11.519850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:03:11.519850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:03:11.519850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:03:11.519850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:03:11.519850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:03:11.519850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:03:11.519850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:03:11.519850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:03:11.519850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:03:11.519850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:03:11.519850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:03:11.519850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:03:11.519850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 583/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:03:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:03:11.525891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7160423-4dd0-47a9-957f-8f8711add6b6 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:11.552385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:11.623420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7160423-4dd0-47a9-957f-8f8711add6b6 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:11.656813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da5cd7-21c8-4c49-b182-291ec408deff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:11.657410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da5cd7-21c8-4c49-b182-291ec408deff None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:11.657692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da5cd7-21c8-4c49-b182-291ec408deff None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:11.657961 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da5cd7-21c8-4c49-b182-291ec408deff None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:11.658615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da5cd7-21c8-4c49-b182-291ec408deff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:03:11.658924 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da5cd7-21c8-4c49-b182-291ec408deff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:11.660271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da5cd7-21c8-4c49-b182-291ec408deff None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:11.668236 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 598/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:03:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:03:11.673993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41f5bc-1b49-497f-b21a-63e1b5b910ee None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:11.770682 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41f5bc-1b49-497f-b21a-63e1b5b910ee None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:11.803087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab464c41-e918-4d90-877f-862c145308fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:11.803700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab464c41-e918-4d90-877f-862c145308fe None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:11.803977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab464c41-e918-4d90-877f-862c145308fe None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:11.804271 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab464c41-e918-4d90-877f-862c145308fe None admin] PATH_INFO: `/v3/users/93ff484df14f4d46a0c11920161f9479` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:11.818054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab464c41-e918-4d90-877f-862c145308fe None admin] RBAC: Authorizing `identity:get_user(user_id=93ff484df14f4d46a0c11920161f9479)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:11.819482 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab464c41-e918-4d90-877f-862c145308fe None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:11.821165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 584/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:03:11 2022] GET /identity/v3/users/93ff484df14f4d46a0c11920161f9479 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:03:11.827724 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da5cd7-21c8-4c49-b182-291ec408deff None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:11.912290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:11.923977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da5cd7-21c8-4c49-b182-291ec408deff None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:11.956963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a474a-85ec-4eeb-9877-39cc117c00d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:11.957603 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a474a-85ec-4eeb-9877-39cc117c00d5 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:11.958018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a474a-85ec-4eeb-9877-39cc117c00d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:11.958434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a474a-85ec-4eeb-9877-39cc117c00d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:12.023427 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a474a-85ec-4eeb-9877-39cc117c00d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:12.025515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a474a-85ec-4eeb-9877-39cc117c00d5 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:12.107940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:12.125787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 599/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:03:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:03:12.131768 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab464c41-e918-4d90-877f-862c145308fe None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:12.179376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:12.227791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab464c41-e918-4d90-877f-862c145308fe None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:12.264582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bcc27-958f-48ce-a278-5a70c167bd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:12.265220 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bcc27-958f-48ce-a278-5a70c167bd9e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:12.265501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bcc27-958f-48ce-a278-5a70c167bd9e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:12.265774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bcc27-958f-48ce-a278-5a70c167bd9e None admin] PATH_INFO: `/v3/projects/0b584cc6c23a4c2e935168e51f99bf8f` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:12.266868 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bcc27-958f-48ce-a278-5a70c167bd9e None admin] RBAC: Authorizing `identity:get_project(project_id=0b584cc6c23a4c2e935168e51f99bf8f)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:12.268277 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bcc27-958f-48ce-a278-5a70c167bd9e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:12.269919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 585/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:03:12 2022] GET /identity/v3/projects/0b584cc6c23a4c2e935168e51f99bf8f => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:03:12.276027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a474a-85ec-4eeb-9877-39cc117c00d5 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:12.370963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a474a-85ec-4eeb-9877-39cc117c00d5 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:12.403168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4f83e-dc5f-4897-96de-e161977e3ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:12.403754 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4f83e-dc5f-4897-96de-e161977e3ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:12.404030 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4f83e-dc5f-4897-96de-e161977e3ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:12.404298 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4f83e-dc5f-4897-96de-e161977e3ed0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:12.405048 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4f83e-dc5f-4897-96de-e161977e3ed0 None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=0b584cc6c23a4c2e935168e51f99bf8f, user.id=93ff484df14f4d46a0c11920161f9479) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:03:12.405354 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4f83e-dc5f-4897-96de-e161977e3ed0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:12.406762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4f83e-dc5f-4897-96de-e161977e3ed0 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:12.411870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 600/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:03:12 2022] GET /identity/v3/role_assignments?user.id=93ff484df14f4d46a0c11920161f9479&scope.project.id=0b584cc6c23a4c2e935168e51f99bf8f&role.id=906e3a8a6c7540a38f629f2567133347 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:03:14.262445 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ccd68-5209-4bb1-8810-4979a20733f0 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:14.263007 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ccd68-5209-4bb1-8810-4979a20733f0 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:14.263007 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ccd68-5209-4bb1-8810-4979a20733f0 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:14.264330 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 586/1186] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:03:14.269356 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fbdbb-3e77-4c5c-82f1-5cd6b7f251a9 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:14.269731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fbdbb-3e77-4c5c-82f1-5cd6b7f251a9 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:14.269908 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fbdbb-3e77-4c5c-82f1-5cd6b7f251a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:14.287912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-ff3fbdbb-3e77-4c5c-82f1-5cd6b7f251a9 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:03:14.289184 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ff3fbdbb-3e77-4c5c-82f1-5cd6b7f251a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:03:14.335371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:14.354353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 601/1187] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:03:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:03:14.624522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ccd68-5209-4bb1-8810-4979a20733f0 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:14.723012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ccd68-5209-4bb1-8810-4979a20733f0 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:14.738565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:14.756003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666d9f8-446a-44c2-8cb7-f4da3f900eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:14.756655 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666d9f8-446a-44c2-8cb7-f4da3f900eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:14.756921 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666d9f8-446a-44c2-8cb7-f4da3f900eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:14.757190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666d9f8-446a-44c2-8cb7-f4da3f900eb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:14.767508 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666d9f8-446a-44c2-8cb7-f4da3f900eb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-0666d9f8-446a-44c2-8cb7-f4da3f900eb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:03:14.772861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:03:14.774040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:03:14.775010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:03:14.775010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:03:14.775010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:03:14.775010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:03:14.775010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:03:14.775010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:03:14.775010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:03:14.775010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:03:14.775010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:03:14.775010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:03:14.775010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:03:14.775010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:03:14.775010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 587/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:03:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:03:14.780331 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fbdbb-3e77-4c5c-82f1-5cd6b7f251a9 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:14.876172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fbdbb-3e77-4c5c-82f1-5cd6b7f251a9 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:14.910255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2cd47-fee7-465c-a224-8dcd0b5d9bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:14.910859 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2cd47-fee7-465c-a224-8dcd0b5d9bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:14.911137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2cd47-fee7-465c-a224-8dcd0b5d9bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:14.911403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2cd47-fee7-465c-a224-8dcd0b5d9bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:14.912054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2cd47-fee7-465c-a224-8dcd0b5d9bd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:03:14.912353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2cd47-fee7-465c-a224-8dcd0b5d9bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:14.913741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2cd47-fee7-465c-a224-8dcd0b5d9bd3 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:14.921451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 602/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:03:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:03:14.927632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666d9f8-446a-44c2-8cb7-f4da3f900eb6 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:15.022155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666d9f8-446a-44c2-8cb7-f4da3f900eb6 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:15.055009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d696ad-575e-4ccd-a9a4-bcf2e9372733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:15.056203 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d696ad-575e-4ccd-a9a4-bcf2e9372733 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:15.056480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d696ad-575e-4ccd-a9a4-bcf2e9372733 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:15.056782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d696ad-575e-4ccd-a9a4-bcf2e9372733 None admin] PATH_INFO: `/v3/users/93ff484df14f4d46a0c11920161f9479` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:15.057899 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d696ad-575e-4ccd-a9a4-bcf2e9372733 None admin] RBAC: Authorizing `identity:get_user(user_id=93ff484df14f4d46a0c11920161f9479)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:15.059329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d696ad-575e-4ccd-a9a4-bcf2e9372733 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:15.060959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 588/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:03:14 2022] GET /identity/v3/users/93ff484df14f4d46a0c11920161f9479 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:03:15.066651 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2cd47-fee7-465c-a224-8dcd0b5d9bd3 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:15.111801 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:15.164287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2cd47-fee7-465c-a224-8dcd0b5d9bd3 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:15.198215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0222eb-a767-4051-8f1f-0138337d6a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:15.198822 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0222eb-a767-4051-8f1f-0138337d6a2f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:15.199101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0222eb-a767-4051-8f1f-0138337d6a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:15.199369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0222eb-a767-4051-8f1f-0138337d6a2f None admin] PATH_INFO: `/v3/projects/0b584cc6c23a4c2e935168e51f99bf8f` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:15.200468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0222eb-a767-4051-8f1f-0138337d6a2f None admin] RBAC: Authorizing `identity:get_project(project_id=0b584cc6c23a4c2e935168e51f99bf8f)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:15.201916 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0222eb-a767-4051-8f1f-0138337d6a2f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:15.203602 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 603/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:03:15 2022] GET /identity/v3/projects/0b584cc6c23a4c2e935168e51f99bf8f => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:03:15.209410 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d696ad-575e-4ccd-a9a4-bcf2e9372733 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:15.254639 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:15.308026 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d696ad-575e-4ccd-a9a4-bcf2e9372733 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:15.339722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61600c-15ca-4660-8337-6877d7c236bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:15.340349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61600c-15ca-4660-8337-6877d7c236bd None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:15.340625 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61600c-15ca-4660-8337-6877d7c236bd None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:15.340922 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61600c-15ca-4660-8337-6877d7c236bd None admin] PATH_INFO: `/v3/projects/0b584cc6c23a4c2e935168e51f99bf8f/users/93ff484df14f4d46a0c11920161f9479/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:15.342790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61600c-15ca-4660-8337-6877d7c236bd None admin] RBAC: Authorizing `identity:create_grant(project_id=0b584cc6c23a4c2e935168e51f99bf8f, user_id=93ff484df14f4d46a0c11920161f9479, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:15.344205 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61600c-15ca-4660-8337-6877d7c236bd None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:15.408076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 589/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:03:15 2022] PUT /identity/v3/projects/0b584cc6c23a4c2e935168e51f99bf8f/users/93ff484df14f4d46a0c11920161f9479/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:03:17.278347 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfd4e1-96fe-45bb-847b-991e15118aa7 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:17.278873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfd4e1-96fe-45bb-847b-991e15118aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:17.278873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfd4e1-96fe-45bb-847b-991e15118aa7 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:17.280232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 604/1193] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:03:17.285417 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809a981-778c-4c33-99f1-51449f4c54c2 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:17.285783 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809a981-778c-4c33-99f1-51449f4c54c2 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:17.285969 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809a981-778c-4c33-99f1-51449f4c54c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:17.303817 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-3809a981-778c-4c33-99f1-51449f4c54c2 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:03:17.305198 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-3809a981-778c-4c33-99f1-51449f4c54c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:03:17.369286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 590/1194] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:03:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:03:17.639545 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfd4e1-96fe-45bb-847b-991e15118aa7 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:17.738027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfd4e1-96fe-45bb-847b-991e15118aa7 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:17.750457 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:17.770179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fac87-4be9-469b-bf38-f6195db0570e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:17.774099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fac87-4be9-469b-bf38-f6195db0570e None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:17.774099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fac87-4be9-469b-bf38-f6195db0570e None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:17.774099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fac87-4be9-469b-bf38-f6195db0570e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:17.782125 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fac87-4be9-469b-bf38-f6195db0570e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-646fac87-4be9-469b-bf38-f6195db0570e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:03:17.786761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:03:17.787818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:03:17.788846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:03:17.788846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:03:17.788846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:03:17.788846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:03:17.788846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:03:17.788846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:03:17.788846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:03:17.788846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:03:17.788846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:03:17.788846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:03:17.788846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:03:17.788846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:03:17.788846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 605/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:03:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:03:17.794480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809a981-778c-4c33-99f1-51449f4c54c2 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:17.827186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:17.891326 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809a981-778c-4c33-99f1-51449f4c54c2 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:17.924648 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f71eb0-8db0-4cdb-aeff-f965c608ed47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:17.925260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f71eb0-8db0-4cdb-aeff-f965c608ed47 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:17.925550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f71eb0-8db0-4cdb-aeff-f965c608ed47 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:17.925824 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f71eb0-8db0-4cdb-aeff-f965c608ed47 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:17.926488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f71eb0-8db0-4cdb-aeff-f965c608ed47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:03:17.926795 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f71eb0-8db0-4cdb-aeff-f965c608ed47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:17.928161 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f71eb0-8db0-4cdb-aeff-f965c608ed47 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:17.935971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 591/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:03:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:03:17.941836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fac87-4be9-469b-bf38-f6195db0570e None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:18.038404 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fac87-4be9-469b-bf38-f6195db0570e None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:18.069643 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1fa282-b3b6-48f1-b417-cc363418c365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:18.070251 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fa282-b3b6-48f1-b417-cc363418c365 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:18.071048 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fa282-b3b6-48f1-b417-cc363418c365 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:18.071313 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fa282-b3b6-48f1-b417-cc363418c365 None admin] PATH_INFO: `/v3/users/93ff484df14f4d46a0c11920161f9479` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:18.072412 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1fa282-b3b6-48f1-b417-cc363418c365 None admin] RBAC: Authorizing `identity:get_user(user_id=93ff484df14f4d46a0c11920161f9479)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:18.073864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1fa282-b3b6-48f1-b417-cc363418c365 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:18.075399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 606/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 12 12:03:17 2022] GET /identity/v3/users/93ff484df14f4d46a0c11920161f9479 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:03:18.081370 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f71eb0-8db0-4cdb-aeff-f965c608ed47 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:18.174283 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:18.179931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f71eb0-8db0-4cdb-aeff-f965c608ed47 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:18.212897 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b27fc9-870e-4277-8a07-c2cdffcd344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:18.213486 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b27fc9-870e-4277-8a07-c2cdffcd344b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:18.213769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b27fc9-870e-4277-8a07-c2cdffcd344b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:18.214043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b27fc9-870e-4277-8a07-c2cdffcd344b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:18.278562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b27fc9-870e-4277-8a07-c2cdffcd344b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:18.279995 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b27fc9-870e-4277-8a07-c2cdffcd344b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:18.368454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:18.374678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 592/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:03:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:03:18.380815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1fa282-b3b6-48f1-b417-cc363418c365 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:18.422681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:18.479391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1fa282-b3b6-48f1-b417-cc363418c365 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:18.511028 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e578cde-63ab-4614-a8af-290870607676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:18.511644 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e578cde-63ab-4614-a8af-290870607676 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:18.511926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e578cde-63ab-4614-a8af-290870607676 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:18.512199 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e578cde-63ab-4614-a8af-290870607676 None admin] PATH_INFO: `/v3/projects/0b584cc6c23a4c2e935168e51f99bf8f` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:18.513284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e578cde-63ab-4614-a8af-290870607676 None admin] RBAC: Authorizing `identity:get_project(project_id=0b584cc6c23a4c2e935168e51f99bf8f)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:18.514714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e578cde-63ab-4614-a8af-290870607676 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:18.516314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 607/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 12 12:03:18 2022] GET /identity/v3/projects/0b584cc6c23a4c2e935168e51f99bf8f => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:03:18.522338 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b27fc9-870e-4277-8a07-c2cdffcd344b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:03:18.617772 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b27fc9-870e-4277-8a07-c2cdffcd344b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:03:18.651224 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5c25a-10e4-445d-ac48-8192e5ad4633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:03:18.651805 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5c25a-10e4-445d-ac48-8192e5ad4633 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:03:18.652086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5c25a-10e4-445d-ac48-8192e5ad4633 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:03:18.652360 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5c25a-10e4-445d-ac48-8192e5ad4633 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:03:18.653112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5c25a-10e4-445d-ac48-8192e5ad4633 None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=0b584cc6c23a4c2e935168e51f99bf8f, user.id=93ff484df14f4d46a0c11920161f9479) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:03:18.653420 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5c25a-10e4-445d-ac48-8192e5ad4633 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:03:18.654837 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5c25a-10e4-445d-ac48-8192e5ad4633 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:03:18.660362 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 593/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:03:18 2022] GET /identity/v3/role_assignments?user.id=93ff484df14f4d46a0c11920161f9479&scope.project.id=0b584cc6c23a4c2e935168e51f99bf8f&role.id=906e3a8a6c7540a38f629f2567133347 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:03:28.507312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=46 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:03:28.657834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=70 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:29.568034 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307d5d9-3d35-414d-97aa-a69b9319e321 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:29.568623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307d5d9-3d35-414d-97aa-a69b9319e321 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:29.568758 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307d5d9-3d35-414d-97aa-a69b9319e321 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:29.570042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 608/1201] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:04:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:04:29.575230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378bcf5c-5991-4f88-b278-b8ae44067a2e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:29.575665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378bcf5c-5991-4f88-b278-b8ae44067a2e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:29.575781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378bcf5c-5991-4f88-b278-b8ae44067a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:29.593904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-378bcf5c-5991-4f88-b278-b8ae44067a2e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:04:29.595122 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-378bcf5c-5991-4f88-b278-b8ae44067a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:04:29.659976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 594/1202] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:04:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:04:29.926408 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307d5d9-3d35-414d-97aa-a69b9319e321 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:30.028321 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307d5d9-3d35-414d-97aa-a69b9319e321 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:30.060851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bb956-38b5-4f32-8ac1-9895ceca24b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:30.061474 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bb956-38b5-4f32-8ac1-9895ceca24b6 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:30.061757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bb956-38b5-4f32-8ac1-9895ceca24b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:30.062031 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bb956-38b5-4f32-8ac1-9895ceca24b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:30.073104 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5bb956-38b5-4f32-8ac1-9895ceca24b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:30.074557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5bb956-38b5-4f32-8ac1-9895ceca24b6 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-6d5bb956-38b5-4f32-8ac1-9895ceca24b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:04:30.088867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:30.090043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:04:30.091317 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:04:30.091317 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:30.091317 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:30.091317 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 609/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:30.095975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378bcf5c-5991-4f88-b278-b8ae44067a2e None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:30.192001 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378bcf5c-5991-4f88-b278-b8ae44067a2e None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:30.220120 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:30.226032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbd597-2425-4975-bce2-f6365e594965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:30.227630 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbd597-2425-4975-bce2-f6365e594965 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:30.227630 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbd597-2425-4975-bce2-f6365e594965 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:30.227630 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbd597-2425-4975-bce2-f6365e594965 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:30.227893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbd597-2425-4975-bce2-f6365e594965 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:30.228172 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbd597-2425-4975-bce2-f6365e594965 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:30.229594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbd597-2425-4975-bce2-f6365e594965 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:30.241589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 595/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:30.247603 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bb956-38b5-4f32-8ac1-9895ceca24b6 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:30.300991 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:30.346523 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bb956-38b5-4f32-8ac1-9895ceca24b6 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:30.378657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2abdf-1239-44f1-a4cd-413f6edaa4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:30.379256 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2abdf-1239-44f1-a4cd-413f6edaa4d9 None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:30.379538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2abdf-1239-44f1-a4cd-413f6edaa4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:30.379940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2abdf-1239-44f1-a4cd-413f6edaa4d9 None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:30.381159 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2abdf-1239-44f1-a4cd-413f6edaa4d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:30.382602 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2abdf-1239-44f1-a4cd-413f6edaa4d9 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:30.487503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 610/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 12 12:04:30 2022] POST /identity/v3/users => generated 251 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:04:32.409463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f525d8-a7dc-4637-bf5c-e1a4a0b0630d None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:32.410110 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f525d8-a7dc-4637-bf5c-e1a4a0b0630d None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:32.410110 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f525d8-a7dc-4637-bf5c-e1a4a0b0630d None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:32.412669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 596/1206] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:04:32.416932 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04879c87-6ea4-43f9-a4a5-92e224700a4f None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:32.417338 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04879c87-6ea4-43f9-a4a5-92e224700a4f None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:32.417493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04879c87-6ea4-43f9-a4a5-92e224700a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:32.435726 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-04879c87-6ea4-43f9-a4a5-92e224700a4f None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:04:32.436931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-04879c87-6ea4-43f9-a4a5-92e224700a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:04:32.502713 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 611/1207] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:04:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:04:32.771603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f525d8-a7dc-4637-bf5c-e1a4a0b0630d None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:32.875475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f525d8-a7dc-4637-bf5c-e1a4a0b0630d None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:32.910976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88931b-0513-421c-bdc3-50fc8b3d4a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:32.911739 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88931b-0513-421c-bdc3-50fc8b3d4a28 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:32.912020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88931b-0513-421c-bdc3-50fc8b3d4a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:32.912290 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88931b-0513-421c-bdc3-50fc8b3d4a28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:32.922917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88931b-0513-421c-bdc3-50fc8b3d4a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-eb88931b-0513-421c-bdc3-50fc8b3d4a28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:04:32.927891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:32.933284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:32.934285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:32.934285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:32.934285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:32.934285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:32.934285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:04:32.934285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:04:32.934285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:04:32.934285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:04:32.934285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:04:32.934285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:04:32.934285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:04:32.934285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:32.934285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 597/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:04:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:32.935172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04879c87-6ea4-43f9-a4a5-92e224700a4f None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:32.974812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:33.033268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04879c87-6ea4-43f9-a4a5-92e224700a4f None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:33.066949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e6798-e60d-4420-8ecf-01dfebe15dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:33.068120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e6798-e60d-4420-8ecf-01dfebe15dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:33.068400 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e6798-e60d-4420-8ecf-01dfebe15dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:33.068703 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e6798-e60d-4420-8ecf-01dfebe15dc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:33.069462 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e6798-e60d-4420-8ecf-01dfebe15dc5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:33.069769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e6798-e60d-4420-8ecf-01dfebe15dc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:33.071147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e6798-e60d-4420-8ecf-01dfebe15dc5 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:33.078878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 612/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:04:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:33.085295 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88931b-0513-421c-bdc3-50fc8b3d4a28 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:33.134942 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:33.182563 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88931b-0513-421c-bdc3-50fc8b3d4a28 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:33.215552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efafeb-d182-41cd-a2c7-c0addcb201fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:33.216165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efafeb-d182-41cd-a2c7-c0addcb201fb None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:33.216448 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efafeb-d182-41cd-a2c7-c0addcb201fb None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:33.216763 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efafeb-d182-41cd-a2c7-c0addcb201fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:33.227136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efafeb-d182-41cd-a2c7-c0addcb201fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:33.228617 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efafeb-d182-41cd-a2c7-c0addcb201fb None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-69efafeb-d182-41cd-a2c7-c0addcb201fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:04:33.243197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:33.245035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:04:33.246150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:04:33.246150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:33.246150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:33.246150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 598/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:33.250528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e6798-e60d-4420-8ecf-01dfebe15dc5 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:33.348522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:33.351297 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e6798-e60d-4420-8ecf-01dfebe15dc5 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:33.384775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa5038-6d5d-49ca-a98c-61069aeb5cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:33.385375 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa5038-6d5d-49ca-a98c-61069aeb5cfe None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:33.385659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa5038-6d5d-49ca-a98c-61069aeb5cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:33.385934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa5038-6d5d-49ca-a98c-61069aeb5cfe None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:33.386571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa5038-6d5d-49ca-a98c-61069aeb5cfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:33.386873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa5038-6d5d-49ca-a98c-61069aeb5cfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:33.388306 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa5038-6d5d-49ca-a98c-61069aeb5cfe None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:33.401252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 613/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:33.407188 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efafeb-d182-41cd-a2c7-c0addcb201fb None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:33.507370 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efafeb-d182-41cd-a2c7-c0addcb201fb None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:33.508932 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:33.540729 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fe5ab-0a41-4575-b8c2-e0d5b5d41888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:33.541341 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fe5ab-0a41-4575-b8c2-e0d5b5d41888 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:33.541617 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fe5ab-0a41-4575-b8c2-e0d5b5d41888 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:33.541885 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fe5ab-0a41-4575-b8c2-e0d5b5d41888 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:33.549346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fe5ab-0a41-4575-b8c2-e0d5b5d41888 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:33.550803 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fe5ab-0a41-4575-b8c2-e0d5b5d41888 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-261fe5ab-0a41-4575-b8c2-e0d5b5d41888 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:04:33.562315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:33.563432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:33.564395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:33.564395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:33.564395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:04:33.564395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:04:33.564395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:04:33.564395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:04:33.564395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:04:33.564395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:04:33.564395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 12 12:04:33.564395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:33.564395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 599/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:33.569947 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa5038-6d5d-49ca-a98c-61069aeb5cfe None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:33.668247 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa5038-6d5d-49ca-a98c-61069aeb5cfe None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:33.701509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf300fa-ec3f-4797-ae2c-2d346c31c1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:33.702098 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf300fa-ec3f-4797-ae2c-2d346c31c1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:33.702380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf300fa-ec3f-4797-ae2c-2d346c31c1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:33.702654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf300fa-ec3f-4797-ae2c-2d346c31c1f4 None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:33.703374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf300fa-ec3f-4797-ae2c-2d346c31c1f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:33.703677 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf300fa-ec3f-4797-ae2c-2d346c31c1f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:33.705156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf300fa-ec3f-4797-ae2c-2d346c31c1f4 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:33.722826 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 614/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Aug 12 12:04:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:33.728818 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fe5ab-0a41-4575-b8c2-e0d5b5d41888 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:33.827599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fe5ab-0a41-4575-b8c2-e0d5b5d41888 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:33.860253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e390989-c013-4f4e-bafa-5948fc2ad086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:33.860874 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e390989-c013-4f4e-bafa-5948fc2ad086 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:33.861155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e390989-c013-4f4e-bafa-5948fc2ad086 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:33.861426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e390989-c013-4f4e-bafa-5948fc2ad086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:33.892590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:33.929509 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e390989-c013-4f4e-bafa-5948fc2ad086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:33.930882 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e390989-c013-4f4e-bafa-5948fc2ad086 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:34.027691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 600/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:04:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:04:34.033612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf300fa-ec3f-4797-ae2c-2d346c31c1f4 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:34.053259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:34.135114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf300fa-ec3f-4797-ae2c-2d346c31c1f4 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:34.167905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972c607-6fa7-4a83-ac51-8a79b7a00451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:34.168516 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972c607-6fa7-4a83-ac51-8a79b7a00451 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:34.168829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972c607-6fa7-4a83-ac51-8a79b7a00451 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:34.169097 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972c607-6fa7-4a83-ac51-8a79b7a00451 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:34.179743 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5972c607-6fa7-4a83-ac51-8a79b7a00451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:34.181161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5972c607-6fa7-4a83-ac51-8a79b7a00451 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-5972c607-6fa7-4a83-ac51-8a79b7a00451 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:04:34.195800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:34.201183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:04:34.202819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:04:34.202819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:34.202819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:34.202819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 615/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:34.203238 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e390989-c013-4f4e-bafa-5948fc2ad086 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:34.264656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:34.303169 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e390989-c013-4f4e-bafa-5948fc2ad086 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:34.337068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1511273-ee08-475b-b818-774557ef0c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:34.337719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1511273-ee08-475b-b818-774557ef0c21 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:34.344032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1511273-ee08-475b-b818-774557ef0c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:34.344032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1511273-ee08-475b-b818-774557ef0c21 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:34.344032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1511273-ee08-475b-b818-774557ef0c21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:34.344032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1511273-ee08-475b-b818-774557ef0c21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:34.344032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1511273-ee08-475b-b818-774557ef0c21 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:34.352233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 601/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:34.358131 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972c607-6fa7-4a83-ac51-8a79b7a00451 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:34.427039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:34.456533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972c607-6fa7-4a83-ac51-8a79b7a00451 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:34.492596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475e196-0634-4fd9-a87b-d763343fd4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:34.493411 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475e196-0634-4fd9-a87b-d763343fd4cf None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:34.493898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475e196-0634-4fd9-a87b-d763343fd4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:34.494586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475e196-0634-4fd9-a87b-d763343fd4cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:34.501246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475e196-0634-4fd9-a87b-d763343fd4cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:34.502674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475e196-0634-4fd9-a87b-d763343fd4cf None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:34.513882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-e475e196-0634-4fd9-a87b-d763343fd4cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:04:34.513882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:34.513882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:34.513882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:34.513882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:34.513882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:34.513882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:34.513882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:34.513882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:34.513882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:34.513882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:34.513882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:34.513882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:04:34.513882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:04:34.513882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:34.519425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:34.520586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:04:34.520586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:04:34.520586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:04:34.520586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:04:34.520586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:04:34.520586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:04:34.520586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:04:34.520586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:34.520586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 616/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:04:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:34.521178 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1511273-ee08-475b-b818-774557ef0c21 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:34.620526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1511273-ee08-475b-b818-774557ef0c21 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:34.641826 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:34.652844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9df134-8f16-46ae-9fe6-e09f55e2a73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:34.653439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9df134-8f16-46ae-9fe6-e09f55e2a73e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:34.653715 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9df134-8f16-46ae-9fe6-e09f55e2a73e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:34.653982 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9df134-8f16-46ae-9fe6-e09f55e2a73e None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:34.654661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9df134-8f16-46ae-9fe6-e09f55e2a73e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:34.654961 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9df134-8f16-46ae-9fe6-e09f55e2a73e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:34.656373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9df134-8f16-46ae-9fe6-e09f55e2a73e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:34.668778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 602/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:04:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:34.675117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475e196-0634-4fd9-a87b-d763343fd4cf None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:34.773925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475e196-0634-4fd9-a87b-d763343fd4cf None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:34.803154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:34.809485 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e1fc6-271c-42e5-8af9-cb948e403189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:34.810085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e1fc6-271c-42e5-8af9-cb948e403189 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:34.810363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e1fc6-271c-42e5-8af9-cb948e403189 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:34.810634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e1fc6-271c-42e5-8af9-cb948e403189 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:34.811410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e1fc6-271c-42e5-8af9-cb948e403189 None admin] RBAC: Adding query filter params (role.id=1416a05bf5ba4de88889c54ff5c58da2, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=5ee8ba21bc184feeaafacab3e69e87bf) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:34.811714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e1fc6-271c-42e5-8af9-cb948e403189 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:34.813177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e1fc6-271c-42e5-8af9-cb948e403189 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:34.818423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 617/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:04:34 2022] GET /identity/v3/role_assignments?user.id=5ee8ba21bc184feeaafacab3e69e87bf&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=1416a05bf5ba4de88889c54ff5c58da2 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:36.692500 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100a6b2-5598-4781-97e7-6edd614c92b8 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:36.693080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100a6b2-5598-4781-97e7-6edd614c92b8 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:36.693080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100a6b2-5598-4781-97e7-6edd614c92b8 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:36.694438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 603/1220] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:04:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:04:36.699697 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420d843-2575-4e15-97b0-b617d63c5e37 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:36.705746 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420d843-2575-4e15-97b0-b617d63c5e37 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:36.705746 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420d843-2575-4e15-97b0-b617d63c5e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:36.718651 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-5420d843-2575-4e15-97b0-b617d63c5e37 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:04:36.719845 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-5420d843-2575-4e15-97b0-b617d63c5e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:04:36.786003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 618/1221] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:04:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:04:37.061973 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100a6b2-5598-4781-97e7-6edd614c92b8 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:37.162813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100a6b2-5598-4781-97e7-6edd614c92b8 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:37.197240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf4d25-705c-456f-8414-751ebffaebef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:37.197860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf4d25-705c-456f-8414-751ebffaebef None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:37.198141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf4d25-705c-456f-8414-751ebffaebef None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:37.198426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf4d25-705c-456f-8414-751ebffaebef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:37.208252 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf4d25-705c-456f-8414-751ebffaebef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-1ddf4d25-705c-456f-8414-751ebffaebef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:04:37.212918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:37.213980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:37.214986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:37.214986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:37.214986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:37.214986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:37.214986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:04:37.214986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:04:37.214986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:04:37.214986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:04:37.214986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:04:37.214986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:04:37.214986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:04:37.214986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:37.214986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 604/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:04:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:37.220248 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420d843-2575-4e15-97b0-b617d63c5e37 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:37.317185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420d843-2575-4e15-97b0-b617d63c5e37 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:37.339656 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:37.350382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ba92e-8ea2-4fd5-bd7b-2c63632f7bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:37.355837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ba92e-8ea2-4fd5-bd7b-2c63632f7bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:37.355837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ba92e-8ea2-4fd5-bd7b-2c63632f7bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:37.355837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ba92e-8ea2-4fd5-bd7b-2c63632f7bb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:37.355837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ba92e-8ea2-4fd5-bd7b-2c63632f7bb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:37.355837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ba92e-8ea2-4fd5-bd7b-2c63632f7bb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:37.355837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ba92e-8ea2-4fd5-bd7b-2c63632f7bb0 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:37.361867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 619/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:04:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:37.368287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf4d25-705c-456f-8414-751ebffaebef None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:37.412298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:37.467566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf4d25-705c-456f-8414-751ebffaebef None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:37.500827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127b3a9-a6f7-4d00-a0d8-1728a20b6e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:37.501443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127b3a9-a6f7-4d00-a0d8-1728a20b6e13 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:37.501722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127b3a9-a6f7-4d00-a0d8-1728a20b6e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:37.501995 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127b3a9-a6f7-4d00-a0d8-1728a20b6e13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:37.511687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b127b3a9-a6f7-4d00-a0d8-1728a20b6e13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:37.513175 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b127b3a9-a6f7-4d00-a0d8-1728a20b6e13 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-b127b3a9-a6f7-4d00-a0d8-1728a20b6e13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:04:37.527481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:37.528575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:04:37.529552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:04:37.529552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:37.529552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:37.529552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 605/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:37.534636 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ba92e-8ea2-4fd5-bd7b-2c63632f7bb0 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:37.634980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ba92e-8ea2-4fd5-bd7b-2c63632f7bb0 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:37.668683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebe145-81a7-405b-abf6-9a7da97eda26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:37.669538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebe145-81a7-405b-abf6-9a7da97eda26 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:37.669817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebe145-81a7-405b-abf6-9a7da97eda26 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:37.670090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebe145-81a7-405b-abf6-9a7da97eda26 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:37.670730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ebe145-81a7-405b-abf6-9a7da97eda26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:37.671032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ebe145-81a7-405b-abf6-9a7da97eda26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:37.672383 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ebe145-81a7-405b-abf6-9a7da97eda26 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:37.684545 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 620/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:37.690371 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127b3a9-a6f7-4d00-a0d8-1728a20b6e13 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:37.781912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:37.789361 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127b3a9-a6f7-4d00-a0d8-1728a20b6e13 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:37.822008 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb12692-dc91-468b-8c90-050179330d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:37.822627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb12692-dc91-468b-8c90-050179330d75 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:37.822910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb12692-dc91-468b-8c90-050179330d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:37.823184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb12692-dc91-468b-8c90-050179330d75 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:37.830472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb12692-dc91-468b-8c90-050179330d75 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:37.831896 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb12692-dc91-468b-8c90-050179330d75 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-1bb12692-dc91-468b-8c90-050179330d75 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:04:37.843524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:37.844769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:37.849542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:37.849542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:37.849542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:04:37.849542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:04:37.849542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:04:37.849542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:04:37.849542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:04:37.849542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:04:37.849542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 12 12:04:37.849542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:37.849542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 606/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:37.851297 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebe145-81a7-405b-abf6-9a7da97eda26 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:37.899229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:37.950097 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebe145-81a7-405b-abf6-9a7da97eda26 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:37.983665 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20be53-003b-4c25-9b31-51383cde9dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:37.984260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20be53-003b-4c25-9b31-51383cde9dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:37.984543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20be53-003b-4c25-9b31-51383cde9dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:37.984849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20be53-003b-4c25-9b31-51383cde9dd3 None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:37.985574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20be53-003b-4c25-9b31-51383cde9dd3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:37.985880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20be53-003b-4c25-9b31-51383cde9dd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:37.987313 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20be53-003b-4c25-9b31-51383cde9dd3 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:38.003757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 621/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Aug 12 12:04:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:38.009714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb12692-dc91-468b-8c90-050179330d75 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:38.109772 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb12692-dc91-468b-8c90-050179330d75 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:38.142977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a043d3-920c-4aef-819b-86dd3cb7c69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:38.143605 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a043d3-920c-4aef-819b-86dd3cb7c69d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:38.143883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a043d3-920c-4aef-819b-86dd3cb7c69d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:38.144154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a043d3-920c-4aef-819b-86dd3cb7c69d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:38.155201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a043d3-920c-4aef-819b-86dd3cb7c69d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:38.156608 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a043d3-920c-4aef-819b-86dd3cb7c69d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-e4a043d3-920c-4aef-819b-86dd3cb7c69d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:04:38.170941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:38.172519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:04:38.173721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:04:38.173721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:38.173721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:38.173721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 607/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:38.178033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20be53-003b-4c25-9b31-51383cde9dd3 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:38.278427 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20be53-003b-4c25-9b31-51383cde9dd3 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:38.280716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:38.311407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44629d17-518e-4022-91e9-b93cf853f5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:38.312002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44629d17-518e-4022-91e9-b93cf853f5cf None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:38.312277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44629d17-518e-4022-91e9-b93cf853f5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:38.312584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44629d17-518e-4022-91e9-b93cf853f5cf None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:38.313247 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44629d17-518e-4022-91e9-b93cf853f5cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:38.313547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44629d17-518e-4022-91e9-b93cf853f5cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:38.314937 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44629d17-518e-4022-91e9-b93cf853f5cf None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:38.326151 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 622/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:04:38.332127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a043d3-920c-4aef-819b-86dd3cb7c69d None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:38.342983 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:38.432544 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a043d3-920c-4aef-819b-86dd3cb7c69d None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:38.466384 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aabbe6-3f70-4b11-8f78-32130e746b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:38.467003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aabbe6-3f70-4b11-8f78-32130e746b80 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:38.467286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aabbe6-3f70-4b11-8f78-32130e746b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:38.467559 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aabbe6-3f70-4b11-8f78-32130e746b80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:38.474084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aabbe6-3f70-4b11-8f78-32130e746b80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:38.475536 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aabbe6-3f70-4b11-8f78-32130e746b80 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:38.486721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-c9aabbe6-3f70-4b11-8f78-32130e746b80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:04:38.486721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:38.486721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:38.486721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:38.486721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:38.486721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:38.486721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:38.486721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:38.486721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:38.486721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:38.486721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:38.486721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:38.486721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:04:38.486721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:04:38.486721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:38.496487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:38.497579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:04:38.497579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:04:38.497579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:04:38.497579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:04:38.497579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:04:38.497579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:04:38.497579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:04:38.497579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:38.497579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 608/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:04:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:38.498090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44629d17-518e-4022-91e9-b93cf853f5cf None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:38.592733 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44629d17-518e-4022-91e9-b93cf853f5cf None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:38.626557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb21e7-14c3-4c18-bd86-0c46e98ecc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:38.627180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb21e7-14c3-4c18-bd86-0c46e98ecc41 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:38.627457 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb21e7-14c3-4c18-bd86-0c46e98ecc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:38.627728 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb21e7-14c3-4c18-bd86-0c46e98ecc41 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:38.628416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb21e7-14c3-4c18-bd86-0c46e98ecc41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:38.628746 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb21e7-14c3-4c18-bd86-0c46e98ecc41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:38.630776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb21e7-14c3-4c18-bd86-0c46e98ecc41 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:38.642718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 623/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:04:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:38.648804 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aabbe6-3f70-4b11-8f78-32130e746b80 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:38.713361 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:38.750089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aabbe6-3f70-4b11-8f78-32130e746b80 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:38.784316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb1ed9-3221-4871-a779-087f4b00d36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:38.784996 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb1ed9-3221-4871-a779-087f4b00d36a None admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:38.785306 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb1ed9-3221-4871-a779-087f4b00d36a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:38.785823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb1ed9-3221-4871-a779-087f4b00d36a None admin] PATH_INFO: `/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/5ee8ba21bc184feeaafacab3e69e87bf/roles/1416a05bf5ba4de88889c54ff5c58da2` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:38.800227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb1ed9-3221-4871-a779-087f4b00d36a None admin] RBAC: Authorizing `identity:create_grant(project_id=20c5570631a34c4ab3868eb1cd96e345, user_id=5ee8ba21bc184feeaafacab3e69e87bf, role_id=1416a05bf5ba4de88889c54ff5c58da2)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:38.802492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb1ed9-3221-4871-a779-087f4b00d36a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:38.871508 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 609/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:04:38 2022] PUT /identity/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/5ee8ba21bc184feeaafacab3e69e87bf/roles/1416a05bf5ba4de88889c54ff5c58da2 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:04:40.742412 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6042c5-0760-4d6f-b8c0-6b0a29dfed90 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:40.742992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6042c5-0760-4d6f-b8c0-6b0a29dfed90 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:40.742992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6042c5-0760-4d6f-b8c0-6b0a29dfed90 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:40.744333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 624/1233] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:04:40.749748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a967782-98e5-4067-aebb-cc6d166a49fa None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:40.750474 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a967782-98e5-4067-aebb-cc6d166a49fa None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:40.750474 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a967782-98e5-4067-aebb-cc6d166a49fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:40.767837 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-2a967782-98e5-4067-aebb-cc6d166a49fa None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:04:40.769056 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2a967782-98e5-4067-aebb-cc6d166a49fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:04:40.836915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 610/1234] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:04:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:04:41.105367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6042c5-0760-4d6f-b8c0-6b0a29dfed90 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:41.130866 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:41.205247 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6042c5-0760-4d6f-b8c0-6b0a29dfed90 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:41.240851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365962f6-7547-4baa-9b3b-1a1cb06f29ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:41.241497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365962f6-7547-4baa-9b3b-1a1cb06f29ee None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:41.241778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365962f6-7547-4baa-9b3b-1a1cb06f29ee None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:41.242050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365962f6-7547-4baa-9b3b-1a1cb06f29ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:41.251911 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365962f6-7547-4baa-9b3b-1a1cb06f29ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-365962f6-7547-4baa-9b3b-1a1cb06f29ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:04:41.256689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:41.257821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:41.258882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:41.258882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:41.258882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:41.258882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:41.258882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:04:41.258882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:04:41.258882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:04:41.258882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:04:41.258882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:04:41.258882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:04:41.258882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 12 12:04:41.258882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:41.258882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 625/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 12 12:04:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:41.263990 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a967782-98e5-4067-aebb-cc6d166a49fa None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:41.309545 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:41.359760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a967782-98e5-4067-aebb-cc6d166a49fa None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:41.392919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fb1ec-d732-4d41-883d-ed7b6a47962d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:41.393503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fb1ec-d732-4d41-883d-ed7b6a47962d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:41.393780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fb1ec-d732-4d41-883d-ed7b6a47962d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:41.394047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fb1ec-d732-4d41-883d-ed7b6a47962d None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:41.395313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344fb1ec-d732-4d41-883d-ed7b6a47962d None admin] RBAC: Adding query filter params (name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:41.395608 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344fb1ec-d732-4d41-883d-ed7b6a47962d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:41.396985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344fb1ec-d732-4d41-883d-ed7b6a47962d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:41.404492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 611/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 12 12:04:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:41.410482 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365962f6-7547-4baa-9b3b-1a1cb06f29ee None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:41.494951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:41.510992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365962f6-7547-4baa-9b3b-1a1cb06f29ee None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:41.544508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae82a4-9246-4afe-8a8c-ec5fed2bf58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:41.545150 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae82a4-9246-4afe-8a8c-ec5fed2bf58d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:41.545426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae82a4-9246-4afe-8a8c-ec5fed2bf58d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:41.545691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae82a4-9246-4afe-8a8c-ec5fed2bf58d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:41.556699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae82a4-9246-4afe-8a8c-ec5fed2bf58d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:41.558140 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae82a4-9246-4afe-8a8c-ec5fed2bf58d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-87ae82a4-9246-4afe-8a8c-ec5fed2bf58d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:04:41.572535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:41.581308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:04:41.582437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:04:41.582437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:41.582437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:41.582437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 626/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:04:41.582720 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fb1ec-d732-4d41-883d-ed7b6a47962d None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:41.678949 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fb1ec-d732-4d41-883d-ed7b6a47962d None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:41.687528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:41.711749 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9abfe-9456-44c5-8c31-21dfea07281d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:41.712338 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9abfe-9456-44c5-8c31-21dfea07281d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:41.712616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9abfe-9456-44c5-8c31-21dfea07281d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:41.712920 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db9abfe-9456-44c5-8c31-21dfea07281d None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:41.713551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9abfe-9456-44c5-8c31-21dfea07281d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:41.713850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9abfe-9456-44c5-8c31-21dfea07281d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:41.715250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db9abfe-9456-44c5-8c31-21dfea07281d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:41.727228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 612/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:41.733104 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae82a4-9246-4afe-8a8c-ec5fed2bf58d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:41.832504 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae82a4-9246-4afe-8a8c-ec5fed2bf58d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:41.865601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e412bec-efc2-4739-bc07-ffacf3a80f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:41.866213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e412bec-efc2-4739-bc07-ffacf3a80f8b None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:41.866487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e412bec-efc2-4739-bc07-ffacf3a80f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:41.866755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e412bec-efc2-4739-bc07-ffacf3a80f8b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:41.869380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:41.874155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e412bec-efc2-4739-bc07-ffacf3a80f8b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:41.875576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e412bec-efc2-4739-bc07-ffacf3a80f8b None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-9e412bec-efc2-4739-bc07-ffacf3a80f8b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:04:41.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:41.891722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:41.892717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:41.892717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:41.892717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:04:41.892717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:04:41.892717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:04:41.892717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:04:41.892717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:04:41.892717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:04:41.892717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 12 12:04:41.892717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:41.892717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 627/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:41.893691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9abfe-9456-44c5-8c31-21dfea07281d None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:41.992984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db9abfe-9456-44c5-8c31-21dfea07281d None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:42.026574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59152bc-72d3-4798-b111-31bba8461871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:42.027167 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59152bc-72d3-4798-b111-31bba8461871 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:42.027447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59152bc-72d3-4798-b111-31bba8461871 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:42.027717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59152bc-72d3-4798-b111-31bba8461871 None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:42.028438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59152bc-72d3-4798-b111-31bba8461871 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:42.028777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59152bc-72d3-4798-b111-31bba8461871 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:42.030196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59152bc-72d3-4798-b111-31bba8461871 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:42.047519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 613/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Aug 12 12:04:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:42.053862 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e412bec-efc2-4739-bc07-ffacf3a80f8b None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:42.154917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e412bec-efc2-4739-bc07-ffacf3a80f8b None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:42.187447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9201f-e47f-4e14-ad50-ea1eff491649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:42.188037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9201f-e47f-4e14-ad50-ea1eff491649 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:42.188345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9201f-e47f-4e14-ad50-ea1eff491649 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:42.188593 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9201f-e47f-4e14-ad50-ea1eff491649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:42.248229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:42.255006 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9201f-e47f-4e14-ad50-ea1eff491649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:42.256440 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9201f-e47f-4e14-ad50-ea1eff491649 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:42.351563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 628/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:04:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:04:42.357389 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59152bc-72d3-4798-b111-31bba8461871 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:42.385744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:42.456709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59152bc-72d3-4798-b111-31bba8461871 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:42.494363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb7aff-a5e4-4f19-b043-a9ed57a8cbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:42.495007 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb7aff-a5e4-4f19-b043-a9ed57a8cbb6 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:42.495287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb7aff-a5e4-4f19-b043-a9ed57a8cbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:42.495554 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb7aff-a5e4-4f19-b043-a9ed57a8cbb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:42.507542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb7aff-a5e4-4f19-b043-a9ed57a8cbb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:42.509169 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb7aff-a5e4-4f19-b043-a9ed57a8cbb6 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-c4cb7aff-a5e4-4f19-b043-a9ed57a8cbb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:04:42.524240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:42.525258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:04:42.526262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:04:42.526262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:42.526262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:42.526262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 614/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:42.531403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9201f-e47f-4e14-ad50-ea1eff491649 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:42.624547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:42.630381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9201f-e47f-4e14-ad50-ea1eff491649 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:42.663898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2325c5b-d7bb-4176-913e-431ff62c725c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:42.664501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2325c5b-d7bb-4176-913e-431ff62c725c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:42.664812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2325c5b-d7bb-4176-913e-431ff62c725c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:42.665089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2325c5b-d7bb-4176-913e-431ff62c725c None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:42.665729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2325c5b-d7bb-4176-913e-431ff62c725c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:42.666029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2325c5b-d7bb-4176-913e-431ff62c725c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:42.667392 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2325c5b-d7bb-4176-913e-431ff62c725c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:42.679965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 629/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:42.685801 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb7aff-a5e4-4f19-b043-a9ed57a8cbb6 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:42.761235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:42.782317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb7aff-a5e4-4f19-b043-a9ed57a8cbb6 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:42.818147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c497e-bee8-40ae-810a-558604f99b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:42.818774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c497e-bee8-40ae-810a-558604f99b41 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:42.819052 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c497e-bee8-40ae-810a-558604f99b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:42.819321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c497e-bee8-40ae-810a-558604f99b41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:42.825859 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c497e-bee8-40ae-810a-558604f99b41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:42.827287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c497e-bee8-40ae-810a-558604f99b41 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:42.838525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-a50c497e-bee8-40ae-810a-558604f99b41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:04:42.838525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:42.838525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:42.838525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:42.838525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:42.838525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:42.838525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:42.838525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:42.838525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:42.838525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:42.838525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:42.838525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:42.838525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:04:42.838525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:04:42.838525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:42.839615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:42.840968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:04:42.840968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:04:42.840968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:04:42.840968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:04:42.840968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:04:42.840968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:04:42.840968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:04:42.840968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:42.840968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 615/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:04:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:04:42.845767 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2325c5b-d7bb-4176-913e-431ff62c725c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:42.945534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2325c5b-d7bb-4176-913e-431ff62c725c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:42.979110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ab174-ca69-4cc3-a182-b6fcd431d369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:42.979709 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ab174-ca69-4cc3-a182-b6fcd431d369 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:42.979990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ab174-ca69-4cc3-a182-b6fcd431d369 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:42.980255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ab174-ca69-4cc3-a182-b6fcd431d369 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:42.980975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ab174-ca69-4cc3-a182-b6fcd431d369 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:42.981277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ab174-ca69-4cc3-a182-b6fcd431d369 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:42.982700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ab174-ca69-4cc3-a182-b6fcd431d369 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:42.995054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 630/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:04:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:43.001158 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c497e-bee8-40ae-810a-558604f99b41 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:43.105389 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c497e-bee8-40ae-810a-558604f99b41 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:43.139883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19e695-7c53-4e2b-b043-27996f6b991c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:43.140476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19e695-7c53-4e2b-b043-27996f6b991c None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:43.140789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19e695-7c53-4e2b-b043-27996f6b991c None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:43.141068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19e695-7c53-4e2b-b043-27996f6b991c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:43.141791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19e695-7c53-4e2b-b043-27996f6b991c None admin] RBAC: Adding query filter params (role.id=1416a05bf5ba4de88889c54ff5c58da2, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=5ee8ba21bc184feeaafacab3e69e87bf) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:43.142101 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19e695-7c53-4e2b-b043-27996f6b991c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:43.143611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19e695-7c53-4e2b-b043-27996f6b991c None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:43.150438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 616/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:04:43 2022] GET /identity/v3/role_assignments?user.id=5ee8ba21bc184feeaafacab3e69e87bf&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=1416a05bf5ba4de88889c54ff5c58da2 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:45.110697 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dee53-03dc-4051-8665-ce516fa2df90 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:45.111247 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dee53-03dc-4051-8665-ce516fa2df90 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:45.111247 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dee53-03dc-4051-8665-ce516fa2df90 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:45.112554 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 631/1247] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:04:45.118364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762003e-d114-4276-8212-6fc8f2a94bc7 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:45.118806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762003e-d114-4276-8212-6fc8f2a94bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:45.118907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762003e-d114-4276-8212-6fc8f2a94bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:45.125257 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:45.137856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-c762003e-d114-4276-8212-6fc8f2a94bc7 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:04:45.139042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c762003e-d114-4276-8212-6fc8f2a94bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:04:45.202691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 617/1248] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:04:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:04:45.470363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dee53-03dc-4051-8665-ce516fa2df90 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:45.489187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:45.571286 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dee53-03dc-4051-8665-ce516fa2df90 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:45.604914 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317994c-ecaa-419e-b3bd-10e772089bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:45.605533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317994c-ecaa-419e-b3bd-10e772089bee None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:45.605814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317994c-ecaa-419e-b3bd-10e772089bee None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:45.606089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317994c-ecaa-419e-b3bd-10e772089bee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:45.616438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317994c-ecaa-419e-b3bd-10e772089bee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-9317994c-ecaa-419e-b3bd-10e772089bee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:04:45.621210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:45.622263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:45.623477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:45.623477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:45.623477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:45.623477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:45.623477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:04:45.623477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:04:45.623477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:04:45.623477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:04:45.623477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:04:45.623477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:04:45.623477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:04:45.623477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:45.623477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 632/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:04:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:45.628552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762003e-d114-4276-8212-6fc8f2a94bc7 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:45.725442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762003e-d114-4276-8212-6fc8f2a94bc7 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:45.753598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:45.758558 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8d2ca-2638-418a-a491-472450ea3d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:45.759155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8d2ca-2638-418a-a491-472450ea3d89 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:45.759441 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8d2ca-2638-418a-a491-472450ea3d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:45.759713 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8d2ca-2638-418a-a491-472450ea3d89 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:45.760399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8d2ca-2638-418a-a491-472450ea3d89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:45.760726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8d2ca-2638-418a-a491-472450ea3d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:45.762106 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c8d2ca-2638-418a-a491-472450ea3d89 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:45.769936 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 618/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:04:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:45.775635 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317994c-ecaa-419e-b3bd-10e772089bee None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:45.854174 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:45.876284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317994c-ecaa-419e-b3bd-10e772089bee None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:45.911693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433722f-56eb-4b06-affc-e769ffbd035b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:45.912321 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433722f-56eb-4b06-affc-e769ffbd035b None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:45.912605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433722f-56eb-4b06-affc-e769ffbd035b None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:45.912916 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433722f-56eb-4b06-affc-e769ffbd035b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:45.923306 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433722f-56eb-4b06-affc-e769ffbd035b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:45.924781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433722f-56eb-4b06-affc-e769ffbd035b None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-c433722f-56eb-4b06-affc-e769ffbd035b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:04:45.939153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:45.940237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:04:45.942060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:04:45.942060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:45.942060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:45.942060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 633/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:45.946674 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8d2ca-2638-418a-a491-472450ea3d89 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:46.047270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8d2ca-2638-418a-a491-472450ea3d89 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:46.081583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c5add-9bde-4376-96ab-94cd03f58a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:46.082176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c5add-9bde-4376-96ab-94cd03f58a8a None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:46.082461 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c5add-9bde-4376-96ab-94cd03f58a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:46.082753 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c5add-9bde-4376-96ab-94cd03f58a8a None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:46.083847 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c5add-9bde-4376-96ab-94cd03f58a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:46.084157 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c5add-9bde-4376-96ab-94cd03f58a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:46.085647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c5add-9bde-4376-96ab-94cd03f58a8a None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:46.097765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 619/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:46.103514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433722f-56eb-4b06-affc-e769ffbd035b None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:46.204914 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433722f-56eb-4b06-affc-e769ffbd035b None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:46.230116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:46.238337 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f33670-2bc0-47ba-aa3a-15e66fddba4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:46.239108 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f33670-2bc0-47ba-aa3a-15e66fddba4d None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:46.239386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f33670-2bc0-47ba-aa3a-15e66fddba4d None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:46.239653 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f33670-2bc0-47ba-aa3a-15e66fddba4d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:46.247144 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f33670-2bc0-47ba-aa3a-15e66fddba4d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:46.248579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f33670-2bc0-47ba-aa3a-15e66fddba4d None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-77f33670-2bc0-47ba-aa3a-15e66fddba4d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:04:46.259477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:46.262945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:46.264101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:46.264101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:46.264101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:04:46.264101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:04:46.264101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:04:46.264101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:04:46.264101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:04:46.264101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:04:46.264101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 12 12:04:46.264101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:46.264101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 634/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:46.266584 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c5add-9bde-4376-96ab-94cd03f58a8a None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:46.321878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:46.365446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c5add-9bde-4376-96ab-94cd03f58a8a None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:46.398988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860c30d-0c80-4a7d-925c-26e82ba49b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:46.399580 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860c30d-0c80-4a7d-925c-26e82ba49b42 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:46.399903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860c30d-0c80-4a7d-925c-26e82ba49b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:46.400190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860c30d-0c80-4a7d-925c-26e82ba49b42 None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:46.401151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860c30d-0c80-4a7d-925c-26e82ba49b42 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:46.401702 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860c30d-0c80-4a7d-925c-26e82ba49b42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:46.403660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860c30d-0c80-4a7d-925c-26e82ba49b42 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:46.421092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 620/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Aug 12 12:04:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:46.427344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f33670-2bc0-47ba-aa3a-15e66fddba4d None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:46.526836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f33670-2bc0-47ba-aa3a-15e66fddba4d None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:46.560464 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964cece-6ccd-4429-9431-9ead978e7778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:46.561086 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964cece-6ccd-4429-9431-9ead978e7778 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:46.561366 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964cece-6ccd-4429-9431-9ead978e7778 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:46.561638 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964cece-6ccd-4429-9431-9ead978e7778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:46.615290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:46.626676 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964cece-6ccd-4429-9431-9ead978e7778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:46.629672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964cece-6ccd-4429-9431-9ead978e7778 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:46.723810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 635/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:04:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 12 12:04:46.729883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860c30d-0c80-4a7d-925c-26e82ba49b42 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:46.828190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860c30d-0c80-4a7d-925c-26e82ba49b42 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:46.842577 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:46.863385 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2bf653-079b-4d2e-88cf-702da42684f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:46.864009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2bf653-079b-4d2e-88cf-702da42684f7 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:46.864317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2bf653-079b-4d2e-88cf-702da42684f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:46.864667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2bf653-079b-4d2e-88cf-702da42684f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:46.878349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2bf653-079b-4d2e-88cf-702da42684f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:46.878349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2bf653-079b-4d2e-88cf-702da42684f7 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-dd2bf653-079b-4d2e-88cf-702da42684f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:04:46.890997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:46.892117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:04:46.893240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:04:46.893240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:46.893240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:46.893240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 621/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:46.898595 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964cece-6ccd-4429-9431-9ead978e7778 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:46.990189 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:47.001481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964cece-6ccd-4429-9431-9ead978e7778 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:47.036100 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da46c14-8229-447b-93b4-4906475cde58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:47.036723 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da46c14-8229-447b-93b4-4906475cde58 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:47.037005 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da46c14-8229-447b-93b4-4906475cde58 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:47.037275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da46c14-8229-447b-93b4-4906475cde58 None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:47.037910 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da46c14-8229-447b-93b4-4906475cde58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:47.038211 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da46c14-8229-447b-93b4-4906475cde58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:47.039570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da46c14-8229-447b-93b4-4906475cde58 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:47.050839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 636/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:04:47.056590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2bf653-079b-4d2e-88cf-702da42684f7 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:47.157689 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2bf653-079b-4d2e-88cf-702da42684f7 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:47.192862 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d918e6-af59-4824-ab97-621d68001dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:47.193478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d918e6-af59-4824-ab97-621d68001dbd None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:47.193783 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d918e6-af59-4824-ab97-621d68001dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:47.194022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d918e6-af59-4824-ab97-621d68001dbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:47.200363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d918e6-af59-4824-ab97-621d68001dbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:47.201823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d918e6-af59-4824-ab97-621d68001dbd None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:47.213222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-29d918e6-af59-4824-ab97-621d68001dbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:04:47.213222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:47.213222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:47.213222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:47.213222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:47.213222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:47.213222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:47.213222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:47.213222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:47.213222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:47.213222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:47.213222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:47.213222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:04:47.213222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:04:47.213222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:47.215925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:47.217027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:04:47.217027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:04:47.217027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:04:47.217027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:04:47.217027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:04:47.217027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:04:47.217027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:04:47.217027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:47.217027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 622/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:04:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:47.220208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da46c14-8229-447b-93b4-4906475cde58 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:47.319032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da46c14-8229-447b-93b4-4906475cde58 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:47.351711 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c9c75-d346-46b8-8d52-cabb283343c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:47.352304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c9c75-d346-46b8-8d52-cabb283343c5 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:47.352586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c9c75-d346-46b8-8d52-cabb283343c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:47.352896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c9c75-d346-46b8-8d52-cabb283343c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:47.353583 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c9c75-d346-46b8-8d52-cabb283343c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:47.353916 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c9c75-d346-46b8-8d52-cabb283343c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:47.355346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c9c75-d346-46b8-8d52-cabb283343c5 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:47.367193 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 637/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:04:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:47.373459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d918e6-af59-4824-ab97-621d68001dbd None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:47.399679 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:47.473552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d918e6-af59-4824-ab97-621d68001dbd None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:47.507804 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa3bfa-3a81-4292-abd5-8a6c3943ee4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:47.508475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa3bfa-3a81-4292-abd5-8a6c3943ee4d None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:47.508785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa3bfa-3a81-4292-abd5-8a6c3943ee4d None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:47.509154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa3bfa-3a81-4292-abd5-8a6c3943ee4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:47.509890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa3bfa-3a81-4292-abd5-8a6c3943ee4d None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=5ee8ba21bc184feeaafacab3e69e87bf) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:47.510186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa3bfa-3a81-4292-abd5-8a6c3943ee4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:47.511602 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa3bfa-3a81-4292-abd5-8a6c3943ee4d None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:47.517092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 623/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:04:47 2022] GET /identity/v3/role_assignments?user.id=5ee8ba21bc184feeaafacab3e69e87bf&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=906e3a8a6c7540a38f629f2567133347 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:49.389123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a604d8-eebb-4834-9d4a-8eea7651d1ff None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:49.389681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a604d8-eebb-4834-9d4a-8eea7651d1ff None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:49.389681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a604d8-eebb-4834-9d4a-8eea7651d1ff None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:49.391006 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 638/1261] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:04:49.396971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c298e-6fff-4c26-8221-755f732c5216 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:49.397382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c298e-6fff-4c26-8221-755f732c5216 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:49.397514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c298e-6fff-4c26-8221-755f732c5216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:49.415429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-0c4c298e-6fff-4c26-8221-755f732c5216 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:04:49.416720 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0c4c298e-6fff-4c26-8221-755f732c5216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:04:49.482458 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 624/1262] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:04:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:04:49.753449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a604d8-eebb-4834-9d4a-8eea7651d1ff None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:49.765070 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:49.855700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a604d8-eebb-4834-9d4a-8eea7651d1ff None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:49.889621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2b5c1-73fe-46a8-9ae7-99b2cdcbf6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:49.890239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2b5c1-73fe-46a8-9ae7-99b2cdcbf6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:49.890522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2b5c1-73fe-46a8-9ae7-99b2cdcbf6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:49.890796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2b5c1-73fe-46a8-9ae7-99b2cdcbf6d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:49.900633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2b5c1-73fe-46a8-9ae7-99b2cdcbf6d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-bff2b5c1-73fe-46a8-9ae7-99b2cdcbf6d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:04:49.905327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:49.906390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:49.915368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:49.915368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:49.915368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:49.915368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:49.915368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:04:49.915368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:04:49.915368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:04:49.915368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:04:49.915368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:04:49.915368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:04:49.915368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:04:49.915368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:49.915368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 639/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:04:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:49.916241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c298e-6fff-4c26-8221-755f732c5216 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:49.931673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:50.009214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c298e-6fff-4c26-8221-755f732c5216 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:50.041653 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f988701-d06a-4eef-a400-d9df62b49793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:50.042245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f988701-d06a-4eef-a400-d9df62b49793 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:50.042526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f988701-d06a-4eef-a400-d9df62b49793 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:50.042800 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f988701-d06a-4eef-a400-d9df62b49793 None admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:50.043709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f988701-d06a-4eef-a400-d9df62b49793 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:50.044014 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f988701-d06a-4eef-a400-d9df62b49793 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:50.045489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f988701-d06a-4eef-a400-d9df62b49793 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:50.054452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 625/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:04:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:50.060361 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2b5c1-73fe-46a8-9ae7-99b2cdcbf6d9 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:50.129105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:50.161064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2b5c1-73fe-46a8-9ae7-99b2cdcbf6d9 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:50.195960 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c09cdd-ecb7-43f3-bdc3-e577f43af189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:50.196585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c09cdd-ecb7-43f3-bdc3-e577f43af189 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:50.196900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c09cdd-ecb7-43f3-bdc3-e577f43af189 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:50.197177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c09cdd-ecb7-43f3-bdc3-e577f43af189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:50.206986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c09cdd-ecb7-43f3-bdc3-e577f43af189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:50.208479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c09cdd-ecb7-43f3-bdc3-e577f43af189 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-85c09cdd-ecb7-43f3-bdc3-e577f43af189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:04:50.222716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:50.225109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:04:50.226488 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:04:50.226488 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:50.226488 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:50.226488 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 640/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:50.230330 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f988701-d06a-4eef-a400-d9df62b49793 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:50.312287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:50.336082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f988701-d06a-4eef-a400-d9df62b49793 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:50.367648 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78358b9-d87d-4365-b895-e3ec257bbba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:50.368233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78358b9-d87d-4365-b895-e3ec257bbba3 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:50.368511 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78358b9-d87d-4365-b895-e3ec257bbba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:50.368812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78358b9-d87d-4365-b895-e3ec257bbba3 None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:50.369445 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78358b9-d87d-4365-b895-e3ec257bbba3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:50.369743 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78358b9-d87d-4365-b895-e3ec257bbba3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:50.371171 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78358b9-d87d-4365-b895-e3ec257bbba3 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:50.383746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 626/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:50.389467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c09cdd-ecb7-43f3-bdc3-e577f43af189 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:50.490689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c09cdd-ecb7-43f3-bdc3-e577f43af189 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:50.510325 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:50.526000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802eff1-8436-40b0-9302-f0c58bd1fdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:50.526622 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802eff1-8436-40b0-9302-f0c58bd1fdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:50.526913 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802eff1-8436-40b0-9302-f0c58bd1fdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:50.527172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802eff1-8436-40b0-9302-f0c58bd1fdf8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:50.534445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802eff1-8436-40b0-9302-f0c58bd1fdf8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:50.535990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802eff1-8436-40b0-9302-f0c58bd1fdf8 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-1802eff1-8436-40b0-9302-f0c58bd1fdf8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:04:50.547050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:50.548170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:50.549419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:50.549419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:50.549419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:04:50.549419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:04:50.549419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:04:50.549419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:04:50.549419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:04:50.549419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:04:50.549419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 12 12:04:50.549419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:50.549419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 641/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:50.554639 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78358b9-d87d-4365-b895-e3ec257bbba3 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:50.650963 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78358b9-d87d-4365-b895-e3ec257bbba3 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:50.684234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92064b-7c05-49f8-821e-1df7bab2b4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:50.684853 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92064b-7c05-49f8-821e-1df7bab2b4bf None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:50.685131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92064b-7c05-49f8-821e-1df7bab2b4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:50.685399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92064b-7c05-49f8-821e-1df7bab2b4bf None admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:50.686112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92064b-7c05-49f8-821e-1df7bab2b4bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:50.686411 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92064b-7c05-49f8-821e-1df7bab2b4bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:50.687832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92064b-7c05-49f8-821e-1df7bab2b4bf None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:50.697255 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:50.705468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 627/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Aug 12 12:04:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:50.711275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802eff1-8436-40b0-9302-f0c58bd1fdf8 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:50.811898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802eff1-8436-40b0-9302-f0c58bd1fdf8 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:50.845264 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb78da8-1db9-49ea-ae7a-45f7d1296b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:50.845873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb78da8-1db9-49ea-ae7a-45f7d1296b4c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:50.846149 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb78da8-1db9-49ea-ae7a-45f7d1296b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:50.846417 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb78da8-1db9-49ea-ae7a-45f7d1296b4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:50.857087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb78da8-1db9-49ea-ae7a-45f7d1296b4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:50.858542 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb78da8-1db9-49ea-ae7a-45f7d1296b4c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-4bb78da8-1db9-49ea-ae7a-45f7d1296b4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:04:50.872774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:50.874029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:04:50.879960 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:04:50.879960 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:50.879960 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:50.879960 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 642/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:50.880231 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92064b-7c05-49f8-821e-1df7bab2b4bf None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:50.979430 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92064b-7c05-49f8-821e-1df7bab2b4bf None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:51.012979 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f56027-9e32-4077-9362-06f7817b0eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:51.013585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f56027-9e32-4077-9362-06f7817b0eca None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:51.013870 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f56027-9e32-4077-9362-06f7817b0eca None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:51.014146 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f56027-9e32-4077-9362-06f7817b0eca None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:51.014782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f56027-9e32-4077-9362-06f7817b0eca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:51.015089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f56027-9e32-4077-9362-06f7817b0eca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:51.016531 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f56027-9e32-4077-9362-06f7817b0eca None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:51.027689 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 628/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:51.033495 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb78da8-1db9-49ea-ae7a-45f7d1296b4c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:51.067332 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:51.133716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb78da8-1db9-49ea-ae7a-45f7d1296b4c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:51.166680 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780e0ad-83e2-41e8-8731-231fd93cfb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:51.167303 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780e0ad-83e2-41e8-8731-231fd93cfb28 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:51.167584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780e0ad-83e2-41e8-8731-231fd93cfb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:51.167857 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780e0ad-83e2-41e8-8731-231fd93cfb28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:51.174839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780e0ad-83e2-41e8-8731-231fd93cfb28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:51.176924 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780e0ad-83e2-41e8-8731-231fd93cfb28 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:51.187763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-8780e0ad-83e2-41e8-8731-231fd93cfb28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:04:51.187763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:51.187763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:51.187763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:51.187763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:51.187763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:51.187763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:51.187763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:51.187763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:51.187763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:51.187763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:51.187763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:51.187763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:04:51.187763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:04:51.187763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:51.188819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:51.189894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:04:51.189894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:04:51.189894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:04:51.189894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:04:51.189894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:04:51.189894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:04:51.189894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:04:51.189894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:51.189894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 643/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:04:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:51.195075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f56027-9e32-4077-9362-06f7817b0eca None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:51.253718 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:51.297429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f56027-9e32-4077-9362-06f7817b0eca None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:51.330678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc118ff-5489-40c8-aa15-9a7f1a9f7d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:51.331275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc118ff-5489-40c8-aa15-9a7f1a9f7d48 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:51.331554 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc118ff-5489-40c8-aa15-9a7f1a9f7d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:51.331828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc118ff-5489-40c8-aa15-9a7f1a9f7d48 None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:51.333439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc118ff-5489-40c8-aa15-9a7f1a9f7d48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:51.333765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc118ff-5489-40c8-aa15-9a7f1a9f7d48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:51.335193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc118ff-5489-40c8-aa15-9a7f1a9f7d48 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:51.347262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 629/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:04:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:51.354218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780e0ad-83e2-41e8-8731-231fd93cfb28 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:51.448355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:51.456509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780e0ad-83e2-41e8-8731-231fd93cfb28 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:51.489015 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb46f5b-3845-4ad3-96b3-4cd0132aabb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:51.489568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb46f5b-3845-4ad3-96b3-4cd0132aabb7 None admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:51.489845 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb46f5b-3845-4ad3-96b3-4cd0132aabb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:51.490138 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb46f5b-3845-4ad3-96b3-4cd0132aabb7 None admin] PATH_INFO: `/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/5ee8ba21bc184feeaafacab3e69e87bf/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:51.492033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb46f5b-3845-4ad3-96b3-4cd0132aabb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=20c5570631a34c4ab3868eb1cd96e345, user_id=5ee8ba21bc184feeaafacab3e69e87bf, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:51.493489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb46f5b-3845-4ad3-96b3-4cd0132aabb7 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:51.566168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 644/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 12 12:04:51 2022] PUT /identity/v3/projects/20c5570631a34c4ab3868eb1cd96e345/users/5ee8ba21bc184feeaafacab3e69e87bf/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:04:53.454297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9a113-3643-4227-ab16-4d4f1f52b63b None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:53.454849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9a113-3643-4227-ab16-4d4f1f52b63b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:53.454849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9a113-3643-4227-ab16-4d4f1f52b63b None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:53.456191 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 630/1274] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:04:53.461505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7208d5b-58ae-46df-a4a4-80164fb2cc8d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:53.461910 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7208d5b-58ae-46df-a4a4-80164fb2cc8d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:53.462127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7208d5b-58ae-46df-a4a4-80164fb2cc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:53.480607 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f7208d5b-58ae-46df-a4a4-80164fb2cc8d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:04:53.481835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f7208d5b-58ae-46df-a4a4-80164fb2cc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:04:53.550517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 645/1275] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:04:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:04:53.821637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9a113-3643-4227-ab16-4d4f1f52b63b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:53.921525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9a113-3643-4227-ab16-4d4f1f52b63b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:53.938110 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:53.954645 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794a5e5-01b8-4bca-8902-97d1edd4cfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:53.955329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794a5e5-01b8-4bca-8902-97d1edd4cfb1 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:53.955606 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794a5e5-01b8-4bca-8902-97d1edd4cfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:53.955875 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794a5e5-01b8-4bca-8902-97d1edd4cfb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:53.966536 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794a5e5-01b8-4bca-8902-97d1edd4cfb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-c794a5e5-01b8-4bca-8902-97d1edd4cfb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 12 12:04:53.971243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:53.972343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:53.973691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:53.973691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:53.973691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:53.973691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:53.973691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 12 12:04:53.973691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 12 12:04:53.973691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 12 12:04:53.973691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 12 12:04:53.973691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 12 12:04:53.973691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 12 12:04:53.973691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 12 12:04:53.973691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:53.973691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 631/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 12 12:04:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:53.978577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7208d5b-58ae-46df-a4a4-80164fb2cc8d None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:54.058172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:54.076181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7208d5b-58ae-46df-a4a4-80164fb2cc8d None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:54.109188 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf41575-651f-4eec-9b8a-81571bff743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:54.109780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf41575-651f-4eec-9b8a-81571bff743c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:54.110059 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf41575-651f-4eec-9b8a-81571bff743c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:54.110345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf41575-651f-4eec-9b8a-81571bff743c None admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:54.111004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf41575-651f-4eec-9b8a-81571bff743c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:54.111301 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf41575-651f-4eec-9b8a-81571bff743c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:54.112683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf41575-651f-4eec-9b8a-81571bff743c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:54.120866 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 646/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 12 12:04:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:54.126409 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794a5e5-01b8-4bca-8902-97d1edd4cfb1 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:54.226659 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794a5e5-01b8-4bca-8902-97d1edd4cfb1 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:54.259911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bd1aa-746a-44b7-aa90-151f03dce837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:54.260519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bd1aa-746a-44b7-aa90-151f03dce837 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:54.260827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bd1aa-746a-44b7-aa90-151f03dce837 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:54.261095 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bd1aa-746a-44b7-aa90-151f03dce837 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:54.270951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932bd1aa-746a-44b7-aa90-151f03dce837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:54.274333 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932bd1aa-746a-44b7-aa90-151f03dce837 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-932bd1aa-746a-44b7-aa90-151f03dce837 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:04:54.286692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:54.287730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:04:54.288937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:04:54.288937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:54.288937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:54.288937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 632/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:04:54.294109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf41575-651f-4eec-9b8a-81571bff743c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:54.392722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf41575-651f-4eec-9b8a-81571bff743c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:54.424987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a87f5-2d26-4eb4-a1e2-88ae1c29728c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:54.426093 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a87f5-2d26-4eb4-a1e2-88ae1c29728c None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:54.426388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a87f5-2d26-4eb4-a1e2-88ae1c29728c None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:54.426659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a87f5-2d26-4eb4-a1e2-88ae1c29728c None admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:54.427341 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a87f5-2d26-4eb4-a1e2-88ae1c29728c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:54.427649 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a87f5-2d26-4eb4-a1e2-88ae1c29728c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:54.429039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a87f5-2d26-4eb4-a1e2-88ae1c29728c None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:54.441036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 647/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:54.447396 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bd1aa-746a-44b7-aa90-151f03dce837 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:54.477686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:54.544954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bd1aa-746a-44b7-aa90-151f03dce837 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:54.578591 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ae152-80ae-4a4c-b8b7-958a1813669e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:54.579195 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ae152-80ae-4a4c-b8b7-958a1813669e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:54.579470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ae152-80ae-4a4c-b8b7-958a1813669e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:54.579737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ae152-80ae-4a4c-b8b7-958a1813669e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:54.587352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ae152-80ae-4a4c-b8b7-958a1813669e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:54.588865 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ae152-80ae-4a4c-b8b7-958a1813669e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-417ae152-80ae-4a4c-b8b7-958a1813669e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 12 12:04:54.600041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:54.601193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:54.602235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:54.602235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:54.602235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 12 12:04:54.602235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 12 12:04:54.602235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 12 12:04:54.602235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 12 12:04:54.602235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 12 12:04:54.602235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 12 12:04:54.602235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 12 12:04:54.602235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:54.602235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 633/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:54.607161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a87f5-2d26-4eb4-a1e2-88ae1c29728c None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:54.611064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:54.706739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a87f5-2d26-4eb4-a1e2-88ae1c29728c None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:54.740485 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061caf9a-49be-4dba-9199-1a5292e38985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:54.741140 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061caf9a-49be-4dba-9199-1a5292e38985 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:54.741417 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061caf9a-49be-4dba-9199-1a5292e38985 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:54.741684 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061caf9a-49be-4dba-9199-1a5292e38985 None admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:54.742469 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061caf9a-49be-4dba-9199-1a5292e38985 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:54.742766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061caf9a-49be-4dba-9199-1a5292e38985 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:54.744173 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061caf9a-49be-4dba-9199-1a5292e38985 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:54.760565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 648/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Aug 12 12:04:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:54.766557 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ae152-80ae-4a4c-b8b7-958a1813669e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:54.857685 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:54.868019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ae152-80ae-4a4c-b8b7-958a1813669e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:54.900422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb183aa-c7d2-4276-8ea0-0c75512d3e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:54.901057 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb183aa-c7d2-4276-8ea0-0c75512d3e3e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:54.901337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb183aa-c7d2-4276-8ea0-0c75512d3e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:54.901610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb183aa-c7d2-4276-8ea0-0c75512d3e3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:54.969493 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb183aa-c7d2-4276-8ea0-0c75512d3e3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:54.970903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb183aa-c7d2-4276-8ea0-0c75512d3e3e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:55.059235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:55.069635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 634/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 12 12:04:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 12 12:04:55.075601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061caf9a-49be-4dba-9199-1a5292e38985 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:55.140439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:55.179690 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061caf9a-49be-4dba-9199-1a5292e38985 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:55.213796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f353d9a-5889-4137-b38f-7d2de07fbc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:55.214419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f353d9a-5889-4137-b38f-7d2de07fbc35 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:55.214701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f353d9a-5889-4137-b38f-7d2de07fbc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:55.214975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f353d9a-5889-4137-b38f-7d2de07fbc35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:55.225494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f353d9a-5889-4137-b38f-7d2de07fbc35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:55.226949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f353d9a-5889-4137-b38f-7d2de07fbc35 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-6f353d9a-5889-4137-b38f-7d2de07fbc35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 12 12:04:55.241921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:55.243042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 12 12:04:55.244117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 12 12:04:55.244117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 12 12:04:55.244117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:55.244117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 649/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:55.249008 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb183aa-c7d2-4276-8ea0-0c75512d3e3e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:55.346368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb183aa-c7d2-4276-8ea0-0c75512d3e3e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:55.380301 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596477a5-c0df-4ad0-bd3d-6c8f6801bf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:55.380915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596477a5-c0df-4ad0-bd3d-6c8f6801bf6b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:55.381194 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596477a5-c0df-4ad0-bd3d-6c8f6801bf6b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:55.381465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596477a5-c0df-4ad0-bd3d-6c8f6801bf6b None admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:55.382171 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596477a5-c0df-4ad0-bd3d-6c8f6801bf6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:55.382474 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596477a5-c0df-4ad0-bd3d-6c8f6801bf6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:55.383834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596477a5-c0df-4ad0-bd3d-6c8f6801bf6b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:55.395380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 635/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 12 12:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:55.401277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f353d9a-5889-4137-b38f-7d2de07fbc35 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:55.498665 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f353d9a-5889-4137-b38f-7d2de07fbc35 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:55.512177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:55.532283 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef23aa9-5ddd-4ed9-8427-ba2730f20ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:55.532920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef23aa9-5ddd-4ed9-8427-ba2730f20ade None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:55.533198 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef23aa9-5ddd-4ed9-8427-ba2730f20ade None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:55.533465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef23aa9-5ddd-4ed9-8427-ba2730f20ade None admin] PATH_INFO: `/v3/projects/service` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:55.539910 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef23aa9-5ddd-4ed9-8427-ba2730f20ade None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:55.541367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef23aa9-5ddd-4ed9-8427-ba2730f20ade None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:55.552448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application [None req-6ef23aa9-5ddd-4ed9-8427-ba2730f20ade None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:04:55.552448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:55.552448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:55.552448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:55.552448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:55.552448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:55.552448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:55.552448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:55.552448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:55.552448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:55.552448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:55.552448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:55.552448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 12 12:04:55.552448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 12 12:04:55.552448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:55.553501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:55.554407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 12 12:04:55.554407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 12 12:04:55.554407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 12 12:04:55.554407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 12 12:04:55.554407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 12 12:04:55.554407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 12 12:04:55.554407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 12 12:04:55.554407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: ERROR keystone.server.flask.application Aug 12 12:04:55.554407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 650/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 12 12:04:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 12 12:04:55.559395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596477a5-c0df-4ad0-bd3d-6c8f6801bf6b None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:55.611771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:55.662242 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596477a5-c0df-4ad0-bd3d-6c8f6801bf6b None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:55.696322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb5c1d-f00a-45db-9168-08abb91dfa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:55.696965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb5c1d-f00a-45db-9168-08abb91dfa3b None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:55.697250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb5c1d-f00a-45db-9168-08abb91dfa3b None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:55.697522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb5c1d-f00a-45db-9168-08abb91dfa3b None admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:55.698214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb5c1d-f00a-45db-9168-08abb91dfa3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:55.698517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb5c1d-f00a-45db-9168-08abb91dfa3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:55.699987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb5c1d-f00a-45db-9168-08abb91dfa3b None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:55.712455 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 636/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 12 12:04:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:55.718592 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef23aa9-5ddd-4ed9-8427-ba2730f20ade None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:55.819665 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef23aa9-5ddd-4ed9-8427-ba2730f20ade None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:55.852838 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c170d-a517-422c-9da2-a2a337dcbb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:55.853436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c170d-a517-422c-9da2-a2a337dcbb9f None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:55.853719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c170d-a517-422c-9da2-a2a337dcbb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:55.853993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c170d-a517-422c-9da2-a2a337dcbb9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:55.854716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c170d-a517-422c-9da2-a2a337dcbb9f None admin] RBAC: Adding query filter params (role.id=906e3a8a6c7540a38f629f2567133347, scope.project.id=20c5570631a34c4ab3868eb1cd96e345, user.id=5ee8ba21bc184feeaafacab3e69e87bf) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:55.855025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c170d-a517-422c-9da2-a2a337dcbb9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:55.856473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c170d-a517-422c-9da2-a2a337dcbb9f None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:55.862105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 651/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 12 12:04:55 2022] GET /identity/v3/role_assignments?user.id=5ee8ba21bc184feeaafacab3e69e87bf&scope.project.id=20c5570631a34c4ab3868eb1cd96e345&role.id=906e3a8a6c7540a38f629f2567133347 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:57.811207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36095d6e-81d4-4754-a2b1-ed67a0f8941b None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:57.811634 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36095d6e-81d4-4754-a2b1-ed67a0f8941b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:57.811762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36095d6e-81d4-4754-a2b1-ed67a0f8941b None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:57.813160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 637/1288] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:04:57.818559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334af1ec-f5b4-4cb5-9f75-4e0c07cca1fd None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:57.818963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334af1ec-f5b4-4cb5-9f75-4e0c07cca1fd None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:57.819114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334af1ec-f5b4-4cb5-9f75-4e0c07cca1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:57.837249 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-334af1ec-f5b4-4cb5-9f75-4e0c07cca1fd None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:04:57.838688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-334af1ec-f5b4-4cb5-9f75-4e0c07cca1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:04:57.868768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:57.904866 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 652/1289] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:04:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:04:58.178784 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36095d6e-81d4-4754-a2b1-ed67a0f8941b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:58.283097 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36095d6e-81d4-4754-a2b1-ed67a0f8941b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:58.296067 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:04:58.320029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16031d-6f13-4eb4-b73f-6eb4bdf69ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:58.320732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16031d-6f13-4eb4-b73f-6eb4bdf69ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:58.320952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16031d-6f13-4eb4-b73f-6eb4bdf69ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:58.321216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16031d-6f13-4eb4-b73f-6eb4bdf69ee0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:58.321786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16031d-6f13-4eb4-b73f-6eb4bdf69ee0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:58.323139 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16031d-6f13-4eb4-b73f-6eb4bdf69ee0 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-0b16031d-6f13-4eb4-b73f-6eb4bdf69ee0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:04:58.334459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:04:58.335775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:04:58.336977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:04:58.336977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:04:58.336977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:04:58.336977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:04:58.336977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:04:58.336977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:04:58.336977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:04:58.336977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:04:58.336977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:04:58.336977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:04:58.336977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:04:58.336977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:04:58.336977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 12:04:58.336977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:04:58.336977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 638/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 12 12:04:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:04:58.342265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334af1ec-f5b4-4cb5-9f75-4e0c07cca1fd None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:58.450296 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334af1ec-f5b4-4cb5-9f75-4e0c07cca1fd None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:58.490125 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47111262-2ac4-4ba2-8202-03e3bd479947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:58.490733 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47111262-2ac4-4ba2-8202-03e3bd479947 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:58.491014 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47111262-2ac4-4ba2-8202-03e3bd479947 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:58.491288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47111262-2ac4-4ba2-8202-03e3bd479947 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:58.491922 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47111262-2ac4-4ba2-8202-03e3bd479947 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:58.492222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47111262-2ac4-4ba2-8202-03e3bd479947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:58.493623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47111262-2ac4-4ba2-8202-03e3bd479947 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:58.498397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 653/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Fri Aug 12 12:04:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:04:58.505281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16031d-6f13-4eb4-b73f-6eb4bdf69ee0 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:04:58.622174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16031d-6f13-4eb4-b73f-6eb4bdf69ee0 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:04:58.656701 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befed33-3ef1-4a4e-bf2c-d55b84e6a9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:04:58.657293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befed33-3ef1-4a4e-bf2c-d55b84e6a9eb None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:04:58.657576 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befed33-3ef1-4a4e-bf2c-d55b84e6a9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:04:58.657845 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befed33-3ef1-4a4e-bf2c-d55b84e6a9eb None admin] PATH_INFO: `/v3/services` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:04:58.658470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befed33-3ef1-4a4e-bf2c-d55b84e6a9eb None admin] RBAC: Adding query filter params (type=placement) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:04:58.658772 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befed33-3ef1-4a4e-bf2c-d55b84e6a9eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:04:58.660149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befed33-3ef1-4a4e-bf2c-d55b84e6a9eb None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:04:58.664590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 639/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Fri Aug 12 12:04:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:05:00.598883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad41e4c-6af6-47aa-93eb-028437d14dbe None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:05:00.599408 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad41e4c-6af6-47aa-93eb-028437d14dbe None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:05:00.599523 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad41e4c-6af6-47aa-93eb-028437d14dbe None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:05:00.600843 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 654/1293] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:05:00.606282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab807bc-fd8b-4d32-bc07-2ddbb0b38f3b None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:05:00.606681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab807bc-fd8b-4d32-bc07-2ddbb0b38f3b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:05:00.606839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab807bc-fd8b-4d32-bc07-2ddbb0b38f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:05:00.625266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-aab807bc-fd8b-4d32-bc07-2ddbb0b38f3b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:05:00.626465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-aab807bc-fd8b-4d32-bc07-2ddbb0b38f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:05:00.655297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:05:00.692825 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 640/1294] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:05:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:05:00.962791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad41e4c-6af6-47aa-93eb-028437d14dbe None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:05:00.993629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:05:01.062098 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad41e4c-6af6-47aa-93eb-028437d14dbe None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:05:01.095267 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf84c37-7a9f-40c9-b18c-d41572952833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:05:01.095948 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf84c37-7a9f-40c9-b18c-d41572952833 None admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:05:01.096244 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf84c37-7a9f-40c9-b18c-d41572952833 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:05:01.096517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf84c37-7a9f-40c9-b18c-d41572952833 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:05:01.097395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf84c37-7a9f-40c9-b18c-d41572952833 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:05:01.098761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf84c37-7a9f-40c9-b18c-d41572952833 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:05:01.165697 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 655/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 12 12:05:00 2022] POST /identity/v3/services => generated 246 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:05:03.083997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ee01a-1846-4d20-bb3e-ab26928709af None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:05:03.084509 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ee01a-1846-4d20-bb3e-ab26928709af None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:05:03.084509 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ee01a-1846-4d20-bb3e-ab26928709af None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:05:03.085896 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 641/1296] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:05:03.091121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723dc80-b9b2-43ab-89b8-9805b76f386f None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:05:03.091546 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723dc80-b9b2-43ab-89b8-9805b76f386f None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:05:03.091650 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723dc80-b9b2-43ab-89b8-9805b76f386f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:05:03.110197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-3723dc80-b9b2-43ab-89b8-9805b76f386f None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:05:03.111379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-3723dc80-b9b2-43ab-89b8-9805b76f386f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:05:03.182397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 656/1297] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:05:03 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 12 12:05:03.448937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ee01a-1846-4d20-bb3e-ab26928709af None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:05:03.547650 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ee01a-1846-4d20-bb3e-ab26928709af None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:05:03.581816 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdcae6-c577-42df-b77c-352867fccc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:05:03.582429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdcae6-c577-42df-b77c-352867fccc7e None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:05:03.582707 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdcae6-c577-42df-b77c-352867fccc7e None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:05:03.582977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdcae6-c577-42df-b77c-352867fccc7e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:05:03.583550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fdcae6-c577-42df-b77c-352867fccc7e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:05:03.584924 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fdcae6-c577-42df-b77c-352867fccc7e None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-58fdcae6-c577-42df-b77c-352867fccc7e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:05:03.593366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:05:03.594432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:05:03.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:05:03.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:05:03.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:05:03.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:05:03.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:05:03.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:05:03.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:05:03.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:05:03.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:05:03.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:05:03.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:05:03.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:05:03.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 12:05:03.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:05:03.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 642/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 12 12:05:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:05:03.601344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723dc80-b9b2-43ab-89b8-9805b76f386f None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:05:03.620331 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:05:03.700015 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723dc80-b9b2-43ab-89b8-9805b76f386f None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:05:03.733648 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58b7d6-932f-4cbd-932a-7cca2b2731c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:05:03.734246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58b7d6-932f-4cbd-932a-7cca2b2731c5 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:05:03.734552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58b7d6-932f-4cbd-932a-7cca2b2731c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:05:03.734825 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58b7d6-932f-4cbd-932a-7cca2b2731c5 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:05:03.735532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58b7d6-932f-4cbd-932a-7cca2b2731c5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:05:03.735832 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58b7d6-932f-4cbd-932a-7cca2b2731c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:05:03.737235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58b7d6-932f-4cbd-932a-7cca2b2731c5 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:05:03.742300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 657/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Fri Aug 12 12:05:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:05:03.748449 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdcae6-c577-42df-b77c-352867fccc7e None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:05:03.782142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:05:03.849983 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdcae6-c577-42df-b77c-352867fccc7e None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:05:03.882746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e40aa-3e81-4a58-aa36-ac0bbf3b3e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:05:03.883337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e40aa-3e81-4a58-aa36-ac0bbf3b3e06 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:05:03.883613 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e40aa-3e81-4a58-aa36-ac0bbf3b3e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:05:03.883887 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e40aa-3e81-4a58-aa36-ac0bbf3b3e06 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:05:03.884577 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e40aa-3e81-4a58-aa36-ac0bbf3b3e06 None admin] RBAC: Adding query filter params (interface=public, service_id=4210569c42ed46a0ad1f940f939ad8e5, region_id=RegionOne) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:05:03.884911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e40aa-3e81-4a58-aa36-ac0bbf3b3e06 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:05:03.886255 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e40aa-3e81-4a58-aa36-ac0bbf3b3e06 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:05:03.891097 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 643/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Aug 12 12:05:03 2022] GET /identity/v3/endpoints?service_id=4210569c42ed46a0ad1f940f939ad8e5&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:05:03.897689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58b7d6-932f-4cbd-932a-7cca2b2731c5 None admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:05:03.978544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:05:03.996246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58b7d6-932f-4cbd-932a-7cca2b2731c5 None admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:05:04.028875 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343749a-fe6c-45e0-a07b-af38c3c14545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:05:04.029467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343749a-fe6c-45e0-a07b-af38c3c14545 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:05:04.029745 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343749a-fe6c-45e0-a07b-af38c3c14545 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:05:04.030015 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343749a-fe6c-45e0-a07b-af38c3c14545 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:05:04.030613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343749a-fe6c-45e0-a07b-af38c3c14545 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:05:04.031962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343749a-fe6c-45e0-a07b-af38c3c14545 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:05:04.039693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 658/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 12 12:05:03 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:05:05.909495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10866d6-d706-48cb-972f-f174381ff615 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:05:05.910011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10866d6-d706-48cb-972f-f174381ff615 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:05:05.910011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10866d6-d706-48cb-972f-f174381ff615 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:05:05.911380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 644/1302] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:05:05.916738 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f25fc4-02ac-4877-a54a-966caeabf241 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:05:05.917417 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f25fc4-02ac-4877-a54a-966caeabf241 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:05:05.917562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f25fc4-02ac-4877-a54a-966caeabf241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:05:05.935638 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-44f25fc4-02ac-4877-a54a-966caeabf241 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:05:05.936900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-44f25fc4-02ac-4877-a54a-966caeabf241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:05:06.001265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 659/1303] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:05:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 12 12:05:06.268966 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10866d6-d706-48cb-972f-f174381ff615 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:05:06.360379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:05:06.367681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10866d6-d706-48cb-972f-f174381ff615 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:05:06.400299 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c9006-7d6d-4c43-91e2-f6134e38cf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:05:06.400966 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c9006-7d6d-4c43-91e2-f6134e38cf24 None admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:05:06.401248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c9006-7d6d-4c43-91e2-f6134e38cf24 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:05:06.401518 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c9006-7d6d-4c43-91e2-f6134e38cf24 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:05:06.402088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c9006-7d6d-4c43-91e2-f6134e38cf24 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:05:06.403443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c9006-7d6d-4c43-91e2-f6134e38cf24 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application [None req-fc5c9006-7d6d-4c43-91e2-f6134e38cf24 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 12 12:05:06.412028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 12 12:05:06.413059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.get_or_create( Aug 12 12:05:06.413901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 12 12:05:06.413901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application with Lock( Aug 12 12:05:06.413901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 12 12:05:06.413901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self._enter() Aug 12 12:05:06.413901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 12 12:05:06.413901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 12 12:05:06.413901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 12 12:05:06.413901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application return self.creator() Aug 12 12:05:06.413901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 12 12:05:06.413901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application created_value = creator( Aug 12 12:05:06.413901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 12 12:05:06.413901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 12 12:05:06.413901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 12 12:05:06.413901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: ERROR keystone.server.flask.application Aug 12 12:05:06.413901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 645/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 12 12:05:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 12 12:05:06.419493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f25fc4-02ac-4877-a54a-966caeabf241 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:05:06.494503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:05:06.523401 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f25fc4-02ac-4877-a54a-966caeabf241 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:05:06.556788 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165facd-5fa8-4b40-8d76-6724f9a8ea31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:05:06.557387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165facd-5fa8-4b40-8d76-6724f9a8ea31 None admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:05:06.557670 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165facd-5fa8-4b40-8d76-6724f9a8ea31 None admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:05:06.557944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165facd-5fa8-4b40-8d76-6724f9a8ea31 None admin] PATH_INFO: `/v3/services` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:05:06.558718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165facd-5fa8-4b40-8d76-6724f9a8ea31 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:05:06.559020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165facd-5fa8-4b40-8d76-6724f9a8ea31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:05:06.560376 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165facd-5fa8-4b40-8d76-6724f9a8ea31 None admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:05:06.567019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 660/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Fri Aug 12 12:05:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:05:06.571756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c9006-7d6d-4c43-91e2-f6134e38cf24 None admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:05:06.670379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c9006-7d6d-4c43-91e2-f6134e38cf24 None admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:05:06.703490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07e3d0-fa6d-4de4-8bc8-aeee7bcd3615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:05:06.704104 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07e3d0-fa6d-4de4-8bc8-aeee7bcd3615 None admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:05:06.704382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07e3d0-fa6d-4de4-8bc8-aeee7bcd3615 None admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:05:06.704663 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07e3d0-fa6d-4de4-8bc8-aeee7bcd3615 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:05:06.705320 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07e3d0-fa6d-4de4-8bc8-aeee7bcd3615 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:05:06.706737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07e3d0-fa6d-4de4-8bc8-aeee7bcd3615 None admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:05:06.810524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 646/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:05:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:05:16.567457 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=30 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:05:16.805953 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:10:48.054840 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ab11b-1057-45c3-b0df-1a9e333d9c93 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:48.055447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ab11b-1057-45c3-b0df-1a9e333d9c93 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:48.055447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ab11b-1057-45c3-b0df-1a9e333d9c93 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:48.062128 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacf185-a121-4345-a6be-1a4110a40f41 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:48.062594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacf185-a121-4345-a6be-1a4110a40f41 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:48.062676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacf185-a121-4345-a6be-1a4110a40f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:48.067498 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 661/1307] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:10:48 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:10:48.164930 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-0bacf185-a121-4345-a6be-1a4110a40f41 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:10:48.164930 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0bacf185-a121-4345-a6be-1a4110a40f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:10:48.397817 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 647/1308] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:10:48.705757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d2c3e-8f57-43d2-a3c0-93772ad98722 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:48.706057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d2c3e-8f57-43d2-a3c0-93772ad98722 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:48.706503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d2c3e-8f57-43d2-a3c0-93772ad98722 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:48.708203 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 662/1309] 10.222.0.27 () {54 vars in 965 bytes} [Fri Aug 12 12:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:10:48.715324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066df9a4-006c-4cab-9745-2948c92b4680 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:48.715873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066df9a4-006c-4cab-9745-2948c92b4680 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:48.723127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066df9a4-006c-4cab-9745-2948c92b4680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:48.798326 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-066df9a4-006c-4cab-9745-2948c92b4680 None None] MFA Rules not processed for user `745fad9c109a4280906eb0349849117d`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:10:48.799919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-066df9a4-006c-4cab-9745-2948c92b4680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:10:48.824294 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:10:48.883818 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 648/1310] 10.222.0.27 () {58 vars in 1062 bytes} [Fri Aug 12 12:10:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:10:48.892947 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d2c3e-8f57-43d2-a3c0-93772ad98722 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:10:48.894903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-192d2c3e-8f57-43d2-a3c0-93772ad98722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:10:49.022228 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d2c3e-8f57-43d2-a3c0-93772ad98722 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:10:49.110324 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db92511-c646-4fc6-8d26-d0417323de11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:10:49.111120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db92511-c646-4fc6-8d26-d0417323de11 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:49.111399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db92511-c646-4fc6-8d26-d0417323de11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:49.111671 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db92511-c646-4fc6-8d26-d0417323de11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:49.113352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9db92511-c646-4fc6-8d26-d0417323de11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:10:49.181574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:10:49.247560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db92511-c646-4fc6-8d26-d0417323de11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:10:49.249057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db92511-c646-4fc6-8d26-d0417323de11 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:10:49.270787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 663/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:10:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:10:51.509868 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e2445-d81c-4b84-ad74-e5eaa8da5186 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:51.510429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e2445-d81c-4b84-ad74-e5eaa8da5186 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:51.510429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e2445-d81c-4b84-ad74-e5eaa8da5186 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:51.511747 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 649/1312] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:10:51.516911 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7800950-b594-4532-b7d1-4056fd9d03d2 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:51.517325 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7800950-b594-4532-b7d1-4056fd9d03d2 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:51.517471 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7800950-b594-4532-b7d1-4056fd9d03d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:51.598778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-e7800950-b594-4532-b7d1-4056fd9d03d2 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:10:51.600037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e7800950-b594-4532-b7d1-4056fd9d03d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:10:51.703673 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 664/1313] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:10:52.001098 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d4d97-e121-47a5-a14b-f9265bc4f37e None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:52.001390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d4d97-e121-47a5-a14b-f9265bc4f37e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:52.001665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d4d97-e121-47a5-a14b-f9265bc4f37e None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:52.003034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 650/1314] 10.222.0.27 () {54 vars in 965 bytes} [Fri Aug 12 12:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:10:52.008789 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade5c76-a667-49fe-907c-a030f08a3cb3 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:52.009103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade5c76-a667-49fe-907c-a030f08a3cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:52.009402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade5c76-a667-49fe-907c-a030f08a3cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:52.021745 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:10:52.085008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-cade5c76-a667-49fe-907c-a030f08a3cb3 None None] MFA Rules not processed for user `745fad9c109a4280906eb0349849117d`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:10:52.086152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-cade5c76-a667-49fe-907c-a030f08a3cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:10:52.161775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 665/1315] 10.222.0.27 () {58 vars in 1062 bytes} [Fri Aug 12 12:10:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:10:52.170849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d4d97-e121-47a5-a14b-f9265bc4f37e None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:10:52.172475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-7a3d4d97-e121-47a5-a14b-f9265bc4f37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:10:52.278689 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d4d97-e121-47a5-a14b-f9265bc4f37e None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:10:52.348526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:10:52.351010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3e4a2-866d-4d0b-beb3-f27754c91d1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:10:52.351608 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3e4a2-866d-4d0b-beb3-f27754c91d1c service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:52.351892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3e4a2-866d-4d0b-beb3-f27754c91d1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:52.352168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3e4a2-866d-4d0b-beb3-f27754c91d1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:52.354758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-7da3e4a2-866d-4d0b-beb3-f27754c91d1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:10:52.463984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3e4a2-866d-4d0b-beb3-f27754c91d1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:10:52.465431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3e4a2-866d-4d0b-beb3-f27754c91d1c service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:10:52.485200 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 651/1316] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:10:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:10:55.144688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9272edc-4629-4b8f-8ddb-09ec3388f938 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:55.145214 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9272edc-4629-4b8f-8ddb-09ec3388f938 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:55.145214 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9272edc-4629-4b8f-8ddb-09ec3388f938 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:55.146504 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 666/1317] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:10:55.152286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1f5e7-0089-468b-b99f-584c794d0d7c None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:55.152735 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1f5e7-0089-468b-b99f-584c794d0d7c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:55.152895 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1f5e7-0089-468b-b99f-584c794d0d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:55.217898 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-21d1f5e7-0089-468b-b99f-584c794d0d7c None None] MFA Rules not processed for user `41b76a7e18b64d968720b907624136d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:10:55.219074 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-21d1f5e7-0089-468b-b99f-584c794d0d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:10:55.268829 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:10:55.312261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 652/1318] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:10:55 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:10:55.596328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9272edc-4629-4b8f-8ddb-09ec3388f938 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:10:55.597911 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b9272edc-4629-4b8f-8ddb-09ec3388f938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:10:55.699284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9272edc-4629-4b8f-8ddb-09ec3388f938 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:10:55.702240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:10:55.771524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834a9f1-a8e2-492a-8fca-b01f8ec77f3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:10:55.772122 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834a9f1-a8e2-492a-8fca-b01f8ec77f3c service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:55.772399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834a9f1-a8e2-492a-8fca-b01f8ec77f3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:55.772690 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834a9f1-a8e2-492a-8fca-b01f8ec77f3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:55.774339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7834a9f1-a8e2-492a-8fca-b01f8ec77f3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:10:55.882453 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834a9f1-a8e2-492a-8fca-b01f8ec77f3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:10:55.883824 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834a9f1-a8e2-492a-8fca-b01f8ec77f3c service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:10:55.902183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 667/1319] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:10:55 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:10:58.568751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86e1f0-b0e9-4f3f-8a51-2e04fcfe9d47 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:58.569289 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86e1f0-b0e9-4f3f-8a51-2e04fcfe9d47 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:58.569386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86e1f0-b0e9-4f3f-8a51-2e04fcfe9d47 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:58.570727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 653/1320] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:10:58.575993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d2f46-f7ef-46e2-93e1-b22e277022b1 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:58.576398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d2f46-f7ef-46e2-93e1-b22e277022b1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:58.576554 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d2f46-f7ef-46e2-93e1-b22e277022b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:58.616777 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:10:58.641195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-0a0d2f46-f7ef-46e2-93e1-b22e277022b1 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:10:58.642386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0a0d2f46-f7ef-46e2-93e1-b22e277022b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:10:58.739958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 668/1321] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:10:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:10:59.024339 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86e1f0-b0e9-4f3f-8a51-2e04fcfe9d47 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:10:59.026443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1a86e1f0-b0e9-4f3f-8a51-2e04fcfe9d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:10:59.129462 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86e1f0-b0e9-4f3f-8a51-2e04fcfe9d47 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:10:59.202968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff541a-35d1-42d1-ad4b-e2e9ea6e1e34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:10:59.203557 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff541a-35d1-42d1-ad4b-e2e9ea6e1e34 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:10:59.203834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff541a-35d1-42d1-ad4b-e2e9ea6e1e34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:10:59.204106 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff541a-35d1-42d1-ad4b-e2e9ea6e1e34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:10:59.205750 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f0ff541a-35d1-42d1-ad4b-e2e9ea6e1e34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:10:59.213807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:10:59.311170 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff541a-35d1-42d1-ad4b-e2e9ea6e1e34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:10:59.312512 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff541a-35d1-42d1-ad4b-e2e9ea6e1e34 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:10:59.330303 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 654/1322] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:10:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:11:02.250993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af9b78-a8a9-4d3d-9b02-c6e47e8937c3 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:02.251551 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af9b78-a8a9-4d3d-9b02-c6e47e8937c3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:02.251641 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af9b78-a8a9-4d3d-9b02-c6e47e8937c3 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:02.253114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 669/1323] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:02.258276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf046b4-c153-4293-bdb9-58d3f6d83628 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:02.258690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf046b4-c153-4293-bdb9-58d3f6d83628 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:02.258830 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf046b4-c153-4293-bdb9-58d3f6d83628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:02.326068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-dcf046b4-c153-4293-bdb9-58d3f6d83628 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:02.327264 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-dcf046b4-c153-4293-bdb9-58d3f6d83628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:02.383408 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:02.422952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 655/1324] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:02.707569 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af9b78-a8a9-4d3d-9b02-c6e47e8937c3 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:11:02.709169 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e2af9b78-a8a9-4d3d-9b02-c6e47e8937c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:02.820808 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af9b78-a8a9-4d3d-9b02-c6e47e8937c3 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:11:02.833147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:02.897111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3f7dd-81ce-4c5a-bad4-3225678c4aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:11:02.897718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3f7dd-81ce-4c5a-bad4-3225678c4aee service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:02.898016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3f7dd-81ce-4c5a-bad4-3225678c4aee service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:02.898277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3f7dd-81ce-4c5a-bad4-3225678c4aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:02.899980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-13f3f7dd-81ce-4c5a-bad4-3225678c4aee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:03.014033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f3f7dd-81ce-4c5a-bad4-3225678c4aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:11:03.015369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f3f7dd-81ce-4c5a-bad4-3225678c4aee service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:11:03.033625 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 670/1325] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:11:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:11:04.994805 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc88783-9fc9-4cc0-bf0f-224dfcca80d2 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:04.995366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc88783-9fc9-4cc0-bf0f-224dfcca80d2 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:04.995366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc88783-9fc9-4cc0-bf0f-224dfcca80d2 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:04.996722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 656/1326] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:11:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:05.002084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8114b-24f0-44bd-93a0-ff6321c17f31 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:05.002497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8114b-24f0-44bd-93a0-ff6321c17f31 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:05.002687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8114b-24f0-44bd-93a0-ff6321c17f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:05.053440 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:05.073069 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-7de8114b-24f0-44bd-93a0-ff6321c17f31 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:05.076321 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7de8114b-24f0-44bd-93a0-ff6321c17f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:05.172037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 671/1327] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:05.454423 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc88783-9fc9-4cc0-bf0f-224dfcca80d2 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:11:05.455958 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2cc88783-9fc9-4cc0-bf0f-224dfcca80d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:05.559677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc88783-9fc9-4cc0-bf0f-224dfcca80d2 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:11:05.637991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd23da-97b6-4e7d-b062-bb06d1f9db9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:11:05.638594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd23da-97b6-4e7d-b062-bb06d1f9db9a service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:05.638871 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd23da-97b6-4e7d-b062-bb06d1f9db9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:05.639139 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd23da-97b6-4e7d-b062-bb06d1f9db9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:05.641065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b5dd23da-97b6-4e7d-b062-bb06d1f9db9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:05.657661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:05.748447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd23da-97b6-4e7d-b062-bb06d1f9db9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:11:05.749873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd23da-97b6-4e7d-b062-bb06d1f9db9a service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:11:05.769358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 657/1328] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:11:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:11:07.901321 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd842a-eb6f-438f-b001-76b4557d13bf None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:07.901802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd842a-eb6f-438f-b001-76b4557d13bf None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:07.901928 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd842a-eb6f-438f-b001-76b4557d13bf None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:07.903191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 672/1329] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:07.908546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a3c6c-faad-4b4c-930e-85fbb2152cf0 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:07.908981 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a3c6c-faad-4b4c-930e-85fbb2152cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:07.909175 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a3c6c-faad-4b4c-930e-85fbb2152cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:07.982761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-2a4a3c6c-faad-4b4c-930e-85fbb2152cf0 None None] MFA Rules not processed for user `41b76a7e18b64d968720b907624136d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:07.983975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2a4a3c6c-faad-4b4c-930e-85fbb2152cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:08.048754 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:08.080463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 658/1330] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:11:07 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:08.383430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd842a-eb6f-438f-b001-76b4557d13bf None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:11:08.385821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9cbd842a-eb6f-438f-b001-76b4557d13bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:08.498593 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd842a-eb6f-438f-b001-76b4557d13bf None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:11:08.518032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:08.573284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5a802-eb9b-472b-80b6-e8cef8966a82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:11:08.573884 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5a802-eb9b-472b-80b6-e8cef8966a82 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:08.574164 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5a802-eb9b-472b-80b6-e8cef8966a82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:08.574437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5a802-eb9b-472b-80b6-e8cef8966a82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:08.576028 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-d0b5a802-eb9b-472b-80b6-e8cef8966a82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:08.685393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5a802-eb9b-472b-80b6-e8cef8966a82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:11:08.686839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5a802-eb9b-472b-80b6-e8cef8966a82 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:11:08.705485 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 673/1331] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:11:08 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:11:11.215524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d29dd2-8818-4564-a001-e7a15de7aab8 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:11.216080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d29dd2-8818-4564-a001-e7a15de7aab8 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:11.216080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d29dd2-8818-4564-a001-e7a15de7aab8 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:11.217440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 659/1332] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:11.222479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf91d97-0e9b-4758-97a2-745baec750e7 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:11.222902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf91d97-0e9b-4758-97a2-745baec750e7 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:11.223041 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf91d97-0e9b-4758-97a2-745baec750e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:11.287567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:11.290918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-cdf91d97-0e9b-4758-97a2-745baec750e7 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:11.292196 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-cdf91d97-0e9b-4758-97a2-745baec750e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:11.390783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 674/1333] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:11:11.669382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d29dd2-8818-4564-a001-e7a15de7aab8 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:11:11.670922 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-79d29dd2-8818-4564-a001-e7a15de7aab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:11.773551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d29dd2-8818-4564-a001-e7a15de7aab8 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:11:11.846861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df6422-f51e-419b-98f8-c7cc67db4591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:11:11.847463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df6422-f51e-419b-98f8-c7cc67db4591 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:11.847745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df6422-f51e-419b-98f8-c7cc67db4591 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:11.848017 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df6422-f51e-419b-98f8-c7cc67db4591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:11.849669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f6df6422-f51e-419b-98f8-c7cc67db4591 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:11.876326 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:11.958659 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df6422-f51e-419b-98f8-c7cc67db4591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:11:11.960080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df6422-f51e-419b-98f8-c7cc67db4591 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:11:11.979139 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 660/1334] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:11:13.932121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80774d3-24ab-430c-9034-fb7189c8839e None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:13.932685 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80774d3-24ab-430c-9034-fb7189c8839e None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:13.932685 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80774d3-24ab-430c-9034-fb7189c8839e None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:13.934016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 675/1335] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:11:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:13.939201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf13da-1d14-42d6-b861-a01c69be28c5 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:13.939635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf13da-1d14-42d6-b861-a01c69be28c5 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:13.939759 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf13da-1d14-42d6-b861-a01c69be28c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:14.005077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-d7cf13da-1d14-42d6-b861-a01c69be28c5 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:14.006272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d7cf13da-1d14-42d6-b861-a01c69be28c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:14.079842 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:14.101213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 661/1336] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:14.390066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80774d3-24ab-430c-9034-fb7189c8839e None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:11:14.391633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f80774d3-24ab-430c-9034-fb7189c8839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:14.495022 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80774d3-24ab-430c-9034-fb7189c8839e None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:11:14.522589 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:14.568076 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646c679-8efb-44fc-8a17-371654b9f759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:11:14.568692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646c679-8efb-44fc-8a17-371654b9f759 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:14.568973 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646c679-8efb-44fc-8a17-371654b9f759 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:14.569239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646c679-8efb-44fc-8a17-371654b9f759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:14.570817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f646c679-8efb-44fc-8a17-371654b9f759 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:14.678329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646c679-8efb-44fc-8a17-371654b9f759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:11:14.680039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646c679-8efb-44fc-8a17-371654b9f759 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:11:14.698138 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 676/1337] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:11:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:11:17.157730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c83ec-de98-40e7-b11a-a6f6964f4487 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:17.158249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c83ec-de98-40e7-b11a-a6f6964f4487 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:17.158346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c83ec-de98-40e7-b11a-a6f6964f4487 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:17.159618 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 662/1338] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:17.165128 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdac52-39dd-4370-ad07-d381060e51f3 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:17.165707 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdac52-39dd-4370-ad07-d381060e51f3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:17.165707 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdac52-39dd-4370-ad07-d381060e51f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:17.229483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-dbfdac52-39dd-4370-ad07-d381060e51f3 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:17.230674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-dbfdac52-39dd-4370-ad07-d381060e51f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:17.238744 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:17.329254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 677/1339] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:11:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:17.612975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c83ec-de98-40e7-b11a-a6f6964f4487 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:11:17.614654 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8d6c83ec-de98-40e7-b11a-a6f6964f4487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:17.722444 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c83ec-de98-40e7-b11a-a6f6964f4487 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:11:17.794312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f5c7f-585c-4248-a04a-e3442d65ad97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:11:17.794908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f5c7f-585c-4248-a04a-e3442d65ad97 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:17.795184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f5c7f-585c-4248-a04a-e3442d65ad97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:17.795453 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f5c7f-585c-4248-a04a-e3442d65ad97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:17.797098 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9f4f5c7f-585c-4248-a04a-e3442d65ad97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:17.833248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:17.904409 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f5c7f-585c-4248-a04a-e3442d65ad97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:11:17.905852 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f5c7f-585c-4248-a04a-e3442d65ad97 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:11:17.924289 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 663/1340] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:11:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:11:22.228564 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc166093-1748-486b-addc-9d5954a7a538 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:22.229163 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc166093-1748-486b-addc-9d5954a7a538 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:22.229163 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc166093-1748-486b-addc-9d5954a7a538 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:22.230461 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 678/1341] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:22.235655 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946755a-1824-43a6-86af-9a5e9b22d5cc None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:22.236080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946755a-1824-43a6-86af-9a5e9b22d5cc None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:22.236189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946755a-1824-43a6-86af-9a5e9b22d5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:22.302878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-3946755a-1824-43a6-86af-9a5e9b22d5cc None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:22.304133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-3946755a-1824-43a6-86af-9a5e9b22d5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:22.395011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:22.406857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 664/1342] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:22.698835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc166093-1748-486b-addc-9d5954a7a538 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:11:22.700400 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-dc166093-1748-486b-addc-9d5954a7a538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:22.807680 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc166093-1748-486b-addc-9d5954a7a538 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:11:22.845391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:22.881735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d564e8-e4fe-4223-aff7-47b178d6a5b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:11:22.882331 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d564e8-e4fe-4223-aff7-47b178d6a5b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:22.882616 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d564e8-e4fe-4223-aff7-47b178d6a5b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:22.882886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d564e8-e4fe-4223-aff7-47b178d6a5b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:22.885252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-15d564e8-e4fe-4223-aff7-47b178d6a5b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:22.999407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d564e8-e4fe-4223-aff7-47b178d6a5b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:11:23.000800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d564e8-e4fe-4223-aff7-47b178d6a5b5 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:11:23.019576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 679/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:11:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:11:25.949728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899775f-7e33-412e-8f3d-6e3b6dce7c1f None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:25.950253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899775f-7e33-412e-8f3d-6e3b6dce7c1f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:25.950253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899775f-7e33-412e-8f3d-6e3b6dce7c1f None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:25.951512 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 665/1344] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:25.956832 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734fa55-c1ae-4900-a30e-80ccbfd1de61 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:25.957651 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734fa55-c1ae-4900-a30e-80ccbfd1de61 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:25.958087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734fa55-c1ae-4900-a30e-80ccbfd1de61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:26.023767 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-0734fa55-c1ae-4900-a30e-80ccbfd1de61 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:26.025058 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0734fa55-c1ae-4900-a30e-80ccbfd1de61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:26.043472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:26.123008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 680/1345] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:11:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:26.409782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899775f-7e33-412e-8f3d-6e3b6dce7c1f None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:11:26.411327 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-e899775f-7e33-412e-8f3d-6e3b6dce7c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:26.518589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899775f-7e33-412e-8f3d-6e3b6dce7c1f None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:11:26.594685 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccf893-4253-4a5f-8f2b-33dc1d903fe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:11:26.595289 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccf893-4253-4a5f-8f2b-33dc1d903fe8 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:26.595568 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccf893-4253-4a5f-8f2b-33dc1d903fe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:26.595842 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccf893-4253-4a5f-8f2b-33dc1d903fe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:26.597527 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1eccf893-4253-4a5f-8f2b-33dc1d903fe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:26.643401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:26.713961 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eccf893-4253-4a5f-8f2b-33dc1d903fe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:11:26.715394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eccf893-4253-4a5f-8f2b-33dc1d903fe8 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:11:26.734886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 666/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:11:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:11:31.118385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a15697-19a0-454c-9e49-9e9881aa5a9a None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:31.118905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a15697-19a0-454c-9e49-9e9881aa5a9a None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:31.119019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a15697-19a0-454c-9e49-9e9881aa5a9a None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:31.120290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 681/1347] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:11:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:31.125759 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2142fe-3574-4576-aae3-1f7eff5876a6 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:31.126238 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2142fe-3574-4576-aae3-1f7eff5876a6 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:31.126333 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2142fe-3574-4576-aae3-1f7eff5876a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:31.198760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-6e2142fe-3574-4576-aae3-1f7eff5876a6 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:31.199976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6e2142fe-3574-4576-aae3-1f7eff5876a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:31.297707 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:31.300479 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 667/1348] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:31.587781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a15697-19a0-454c-9e49-9e9881aa5a9a None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:11:31.589368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-23a15697-19a0-454c-9e49-9e9881aa5a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:31.694298 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a15697-19a0-454c-9e49-9e9881aa5a9a None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:11:31.742683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:31.768157 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92bdb2e-5c8e-43f0-8077-a6e02ba76e68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:11:31.768776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bdb2e-5c8e-43f0-8077-a6e02ba76e68 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:31.769055 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bdb2e-5c8e-43f0-8077-a6e02ba76e68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:31.769322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bdb2e-5c8e-43f0-8077-a6e02ba76e68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:31.770902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b92bdb2e-5c8e-43f0-8077-a6e02ba76e68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:31.878889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92bdb2e-5c8e-43f0-8077-a6e02ba76e68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:11:31.880329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92bdb2e-5c8e-43f0-8077-a6e02ba76e68 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:11:31.899409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 682/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:11:36.792929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6d878-6d81-42ac-b229-31381d8beb21 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:36.792929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6d878-6d81-42ac-b229-31381d8beb21 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:36.793538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6d878-6d81-42ac-b229-31381d8beb21 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:36.801759 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 668/1350] 10.222.0.27 () {54 vars in 921 bytes} [Fri Aug 12 12:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:36.809737 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ec970-ac1d-4f44-89c5-f1769b57b1b1 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:36.809737 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ec970-ac1d-4f44-89c5-f1769b57b1b1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:36.810226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ec970-ac1d-4f44-89c5-f1769b57b1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:36.906087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-2d7ec970-ac1d-4f44-89c5-f1769b57b1b1 None None] MFA Rules not processed for user `5ee8ba21bc184feeaafacab3e69e87bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:36.907555 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2d7ec970-ac1d-4f44-89c5-f1769b57b1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:36.939708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:37.074744 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 683/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Aug 12 12:11:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:41.453696 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f4660-754c-4d33-a969-b38c1828d6d4 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:41.454232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f4660-754c-4d33-a969-b38c1828d6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:41.454232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f4660-754c-4d33-a969-b38c1828d6d4 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:41.456065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 669/1352] 10.222.0.27 () {54 vars in 921 bytes} [Fri Aug 12 12:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:41.462576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c61b8b-e00c-449c-871a-eed406234bdd None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:41.470017 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c61b8b-e00c-449c-871a-eed406234bdd None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:41.470017 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c61b8b-e00c-449c-871a-eed406234bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:41.565234 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-b7c61b8b-e00c-449c-871a-eed406234bdd None None] MFA Rules not processed for user `5ee8ba21bc184feeaafacab3e69e87bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:41.567335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b7c61b8b-e00c-449c-871a-eed406234bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:41.646891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:41.687282 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 684/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Aug 12 12:11:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:42.773083 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6aa0c-a421-4b8d-b9e4-7f3fd395c88d None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:42.773652 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6aa0c-a421-4b8d-b9e4-7f3fd395c88d None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:42.773765 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6aa0c-a421-4b8d-b9e4-7f3fd395c88d None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:42.775061 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 670/1354] 10.222.0.27 () {54 vars in 921 bytes} [Fri Aug 12 12:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:11:42.780082 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0673e8-55f1-44dc-a8aa-06b1dabf958c None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:42.780508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0673e8-55f1-44dc-a8aa-06b1dabf958c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:42.780655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0673e8-55f1-44dc-a8aa-06b1dabf958c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:42.862649 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-5a0673e8-55f1-44dc-a8aa-06b1dabf958c None None] MFA Rules not processed for user `5ee8ba21bc184feeaafacab3e69e87bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:42.863841 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-5a0673e8-55f1-44dc-a8aa-06b1dabf958c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:42.971282 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 685/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Aug 12 12:11:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:46.313386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84710708-21eb-4721-bad1-3b2842fa6e3f None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:46.314250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84710708-21eb-4721-bad1-3b2842fa6e3f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:46.314538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84710708-21eb-4721-bad1-3b2842fa6e3f None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:46.316250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 671/1356] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:11:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:46.330175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ce624-6c9a-470b-b09c-2ccc536ac29c None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:46.330608 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ce624-6c9a-470b-b09c-2ccc536ac29c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:46.330725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ce624-6c9a-470b-b09c-2ccc536ac29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:46.372592 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:46.416893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-b15ce624-6c9a-470b-b09c-2ccc536ac29c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:46.418115 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b15ce624-6c9a-470b-b09c-2ccc536ac29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:46.545802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 686/1357] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:46.857303 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29493552-a0e7-43d3-8e08-5902471c593a None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:46.857826 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29493552-a0e7-43d3-8e08-5902471c593a None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:46.858253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29493552-a0e7-43d3-8e08-5902471c593a None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:46.860025 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 672/1358] 10.222.0.27 () {54 vars in 962 bytes} [Fri Aug 12 12:11:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:46.867539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224f7fb-846f-42a6-97fe-2630a702beb3 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:46.868135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224f7fb-846f-42a6-97fe-2630a702beb3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:46.868580 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224f7fb-846f-42a6-97fe-2630a702beb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:46.952903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-a224f7fb-846f-42a6-97fe-2630a702beb3 None None] MFA Rules not processed for user `687d0375abd44d2481f988c6df90fad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:46.955090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a224f7fb-846f-42a6-97fe-2630a702beb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:47.009193 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:47.079821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 687/1359] 10.222.0.27 () {58 vars in 1059 bytes} [Fri Aug 12 12:11:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:11:47.088266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29493552-a0e7-43d3-8e08-5902471c593a None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:11:47.089823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-29493552-a0e7-43d3-8e08-5902471c593a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:47.253964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29493552-a0e7-43d3-8e08-5902471c593a None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:11:47.353035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:47.395127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431debb4-2fe2-40a6-a8a0-bed86ed77be9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:11:47.395861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431debb4-2fe2-40a6-a8a0-bed86ed77be9 service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:47.396242 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431debb4-2fe2-40a6-a8a0-bed86ed77be9 service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:47.396634 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431debb4-2fe2-40a6-a8a0-bed86ed77be9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:47.398810 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-431debb4-2fe2-40a6-a8a0-bed86ed77be9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:47.489130 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431debb4-2fe2-40a6-a8a0-bed86ed77be9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:11:47.490871 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431debb4-2fe2-40a6-a8a0-bed86ed77be9 service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:11:47.511632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 673/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:11:49.626961 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3ea63-010b-4a66-aa89-507644af3cf7 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:49.627534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3ea63-010b-4a66-aa89-507644af3cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:49.627534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3ea63-010b-4a66-aa89-507644af3cf7 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:49.628807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 688/1361] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:49.634655 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a479e-c542-4af2-adff-0eab4e1c7266 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:49.635117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a479e-c542-4af2-adff-0eab4e1c7266 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:49.635227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a479e-c542-4af2-adff-0eab4e1c7266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:49.705677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-ff2a479e-c542-4af2-adff-0eab4e1c7266 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:49.706968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-ff2a479e-c542-4af2-adff-0eab4e1c7266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:49.745856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:49.827759 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 674/1362] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:50.125537 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3ea63-010b-4a66-aa89-507644af3cf7 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:11:50.126521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e4d3ea63-010b-4a66-aa89-507644af3cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:50.285210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3ea63-010b-4a66-aa89-507644af3cf7 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:11:50.296510 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:50.409773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444363c0-236a-43cc-a740-8d073f9fae94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:11:50.410723 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444363c0-236a-43cc-a740-8d073f9fae94 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:50.411111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444363c0-236a-43cc-a740-8d073f9fae94 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:50.411460 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444363c0-236a-43cc-a740-8d073f9fae94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:50.414401 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-444363c0-236a-43cc-a740-8d073f9fae94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:50.516313 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444363c0-236a-43cc-a740-8d073f9fae94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:11:50.517952 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444363c0-236a-43cc-a740-8d073f9fae94 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:11:50.537581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 689/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:11:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:11:52.906797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacd0c3-c788-4439-81ff-5779b989f7c3 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:52.907391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacd0c3-c788-4439-81ff-5779b989f7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:52.907391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacd0c3-c788-4439-81ff-5779b989f7c3 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:52.908739 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 675/1364] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:52.914976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0ef1a-4e19-4be6-93da-fac06894d3e8 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:52.915423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0ef1a-4e19-4be6-93da-fac06894d3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:52.915538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0ef1a-4e19-4be6-93da-fac06894d3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:52.932288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:52.985060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-cfc0ef1a-4e19-4be6-93da-fac06894d3e8 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:52.986277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-cfc0ef1a-4e19-4be6-93da-fac06894d3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:53.089336 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 690/1365] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:53.386682 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238598d-0698-493f-ae54-621cc2f6c5e2 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:53.386989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238598d-0698-493f-ae54-621cc2f6c5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:53.387340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238598d-0698-493f-ae54-621cc2f6c5e2 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:53.388762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 676/1366] 10.222.0.27 () {54 vars in 962 bytes} [Fri Aug 12 12:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:53.394258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458af59-829a-4e3a-af0f-e08226a2fec1 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:53.397934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458af59-829a-4e3a-af0f-e08226a2fec1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:53.397934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458af59-829a-4e3a-af0f-e08226a2fec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:53.477434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-3458af59-829a-4e3a-af0f-e08226a2fec1 None None] MFA Rules not processed for user `687d0375abd44d2481f988c6df90fad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:53.478952 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-3458af59-829a-4e3a-af0f-e08226a2fec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:53.517422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:53.607380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 691/1367] 10.222.0.27 () {58 vars in 1059 bytes} [Fri Aug 12 12:11:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:53.615356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238598d-0698-493f-ae54-621cc2f6c5e2 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:11:53.617622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0238598d-0698-493f-ae54-621cc2f6c5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:53.783192 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238598d-0698-493f-ae54-621cc2f6c5e2 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:11:53.811059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:53.912031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76293a15-5ee7-4a9a-b3cc-fbd5857fc3a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:11:53.913276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76293a15-5ee7-4a9a-b3cc-fbd5857fc3a6 service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:53.913515 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76293a15-5ee7-4a9a-b3cc-fbd5857fc3a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:53.913734 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76293a15-5ee7-4a9a-b3cc-fbd5857fc3a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:53.915688 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-76293a15-5ee7-4a9a-b3cc-fbd5857fc3a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:54.021346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76293a15-5ee7-4a9a-b3cc-fbd5857fc3a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:11:54.023517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76293a15-5ee7-4a9a-b3cc-fbd5857fc3a6 service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:11:54.042382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 677/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:11:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:11:54.526040 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c39343-d258-4a9d-898d-1e5c80839d88 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:54.526569 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c39343-d258-4a9d-898d-1e5c80839d88 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:54.526569 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c39343-d258-4a9d-898d-1e5c80839d88 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:54.527917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 692/1369] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:54.533239 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ebc39-2290-44f0-bc98-59d13fdc5147 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:54.533532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ebc39-2290-44f0-bc98-59d13fdc5147 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:54.533810 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ebc39-2290-44f0-bc98-59d13fdc5147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:54.568484 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:54.608785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-f86ebc39-2290-44f0-bc98-59d13fdc5147 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:54.610039 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f86ebc39-2290-44f0-bc98-59d13fdc5147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:54.713251 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 678/1370] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:55.339100 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e92d9-9864-4ce0-a896-50df5824f79b None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:55.339384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e92d9-9864-4ce0-a896-50df5824f79b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:55.339692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e92d9-9864-4ce0-a896-50df5824f79b None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:55.341215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 693/1371] 10.222.0.27 () {54 vars in 964 bytes} [Fri Aug 12 12:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:55.346353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62ad2a-064d-4749-9b19-cacaced20e26 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:55.347219 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62ad2a-064d-4749-9b19-cacaced20e26 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:55.347693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62ad2a-064d-4749-9b19-cacaced20e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:55.436256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-6a62ad2a-064d-4749-9b19-cacaced20e26 None None] MFA Rules not processed for user `935ab3a263ae46fcb929029a147a1010`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:55.437759 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6a62ad2a-064d-4749-9b19-cacaced20e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:55.486029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:55.518181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 679/1372] 10.222.0.27 () {58 vars in 1061 bytes} [Fri Aug 12 12:11:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:11:55.527240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e92d9-9864-4ce0-a896-50df5824f79b None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:11:55.528848 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-492e92d9-9864-4ce0-a896-50df5824f79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:55.644266 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e92d9-9864-4ce0-a896-50df5824f79b None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:11:55.683792 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:55.717674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db20bbb-9adc-4b64-9dc9-e3817a5c08f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:11:55.718281 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db20bbb-9adc-4b64-9dc9-e3817a5c08f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:55.718562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db20bbb-9adc-4b64-9dc9-e3817a5c08f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:55.718902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db20bbb-9adc-4b64-9dc9-e3817a5c08f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:55.720489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-8db20bbb-9adc-4b64-9dc9-e3817a5c08f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:55.873675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db20bbb-9adc-4b64-9dc9-e3817a5c08f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:11:55.875483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db20bbb-9adc-4b64-9dc9-e3817a5c08f8 service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:11:55.896512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 694/1373] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:11:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:11:56.207594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577a5b1-2cc0-48d3-be3d-353eb44d8e16 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:56.207881 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577a5b1-2cc0-48d3-be3d-353eb44d8e16 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:56.208154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577a5b1-2cc0-48d3-be3d-353eb44d8e16 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:56.210173 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 680/1374] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:11:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:56.215592 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a522851-7806-4281-9cc2-eafbba88cf5a None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:56.215942 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a522851-7806-4281-9cc2-eafbba88cf5a None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:56.216298 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a522851-7806-4281-9cc2-eafbba88cf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:56.280733 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-7a522851-7806-4281-9cc2-eafbba88cf5a None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:56.281930 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7a522851-7806-4281-9cc2-eafbba88cf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:56.301901 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:56.376880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 695/1375] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:56.665789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577a5b1-2cc0-48d3-be3d-353eb44d8e16 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:11:56.667491 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8577a5b1-2cc0-48d3-be3d-353eb44d8e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:56.814919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577a5b1-2cc0-48d3-be3d-353eb44d8e16 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:11:56.866637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:56.926937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc55f62-1e6d-4119-a184-e23ad5952086 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:11:56.927543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc55f62-1e6d-4119-a184-e23ad5952086 service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:56.927822 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc55f62-1e6d-4119-a184-e23ad5952086 service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:56.928135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc55f62-1e6d-4119-a184-e23ad5952086 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:56.930112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9dc55f62-1e6d-4119-a184-e23ad5952086 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:57.018202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc55f62-1e6d-4119-a184-e23ad5952086 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:11:57.019775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc55f62-1e6d-4119-a184-e23ad5952086 service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:11:57.038525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 681/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:11:59.110937 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f503464-5b4c-4963-9586-d2beae96fb34 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:59.111460 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f503464-5b4c-4963-9586-d2beae96fb34 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:59.111460 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f503464-5b4c-4963-9586-d2beae96fb34 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:59.113124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 696/1377] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:11:59.118908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0d3e2-5651-4534-bed5-d6696591014f None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:59.119748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0d3e2-5651-4534-bed5-d6696591014f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:59.120180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0d3e2-5651-4534-bed5-d6696591014f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:59.196851 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-72b0d3e2-5651-4534-bed5-d6696591014f None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:11:59.198098 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-72b0d3e2-5651-4534-bed5-d6696591014f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:59.204873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:59.295036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 682/1378] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:11:59.594062 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f503464-5b4c-4963-9586-d2beae96fb34 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:11:59.595678 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4f503464-5b4c-4963-9586-d2beae96fb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:59.760017 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f503464-5b4c-4963-9586-d2beae96fb34 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:11:59.763960 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:11:59.885084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34d269-577a-4219-b6bb-ed40df4bd56b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:11:59.885682 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34d269-577a-4219-b6bb-ed40df4bd56b service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:11:59.885958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34d269-577a-4219-b6bb-ed40df4bd56b service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:11:59.886227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34d269-577a-4219-b6bb-ed40df4bd56b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:11:59.888094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ee34d269-577a-4219-b6bb-ed40df4bd56b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:11:59.984074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34d269-577a-4219-b6bb-ed40df4bd56b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:11:59.985748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34d269-577a-4219-b6bb-ed40df4bd56b service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:00.005609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 697/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:02.081000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc5d7a-000a-4b08-bab8-9dfe7a03448f None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:02.081509 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc5d7a-000a-4b08-bab8-9dfe7a03448f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:02.081727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc5d7a-000a-4b08-bab8-9dfe7a03448f None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:02.083331 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 683/1380] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:02.089342 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66598a10-b955-407c-a0f3-cd67abee558a None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:02.089792 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66598a10-b955-407c-a0f3-cd67abee558a None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:02.089968 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66598a10-b955-407c-a0f3-cd67abee558a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:02.101827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:02.162465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-66598a10-b955-407c-a0f3-cd67abee558a None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:02.163918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-66598a10-b955-407c-a0f3-cd67abee558a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:02.269935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 698/1381] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:02.563385 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc5d7a-000a-4b08-bab8-9dfe7a03448f None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:02.565163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b1bc5d7a-000a-4b08-bab8-9dfe7a03448f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:02.727349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:02.738845 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc5d7a-000a-4b08-bab8-9dfe7a03448f None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:02.875094 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efa695-5cbd-4ed4-a0bf-89a813c04e76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:02.875703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efa695-5cbd-4ed4-a0bf-89a813c04e76 service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:02.875984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efa695-5cbd-4ed4-a0bf-89a813c04e76 service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:02.876255 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efa695-5cbd-4ed4-a0bf-89a813c04e76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:02.877915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-04efa695-5cbd-4ed4-a0bf-89a813c04e76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:02.977190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efa695-5cbd-4ed4-a0bf-89a813c04e76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:02.978794 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efa695-5cbd-4ed4-a0bf-89a813c04e76 service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:02.989944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:02.999888 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 684/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:05.075437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892787c0-0d95-4633-8c0f-4e202de5ef7a None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:05.076083 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892787c0-0d95-4633-8c0f-4e202de5ef7a None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:05.076083 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892787c0-0d95-4633-8c0f-4e202de5ef7a None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:05.077235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 699/1383] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:05.082752 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7c169-5a85-443c-b975-3cfb09757c6f None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:05.083232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7c169-5a85-443c-b975-3cfb09757c6f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:05.083399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7c169-5a85-443c-b975-3cfb09757c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:05.149852 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-bdb7c169-5a85-443c-b975-3cfb09757c6f None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:05.151077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-bdb7c169-5a85-443c-b975-3cfb09757c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:05.251622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 685/1384] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:05.615172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892787c0-0d95-4633-8c0f-4e202de5ef7a None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:05.617232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-892787c0-0d95-4633-8c0f-4e202de5ef7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:05.720062 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:05.784373 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892787c0-0d95-4633-8c0f-4e202de5ef7a None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:05.913995 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0fe5e-334f-46e0-ab11-6e4bace72bc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:05.914602 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0fe5e-334f-46e0-ab11-6e4bace72bc4 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:05.914873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0fe5e-334f-46e0-ab11-6e4bace72bc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:05.915142 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0fe5e-334f-46e0-ab11-6e4bace72bc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:05.916785 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7fa0fe5e-334f-46e0-ab11-6e4bace72bc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:05.967824 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:06.009326 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0fe5e-334f-46e0-ab11-6e4bace72bc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:06.011009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0fe5e-334f-46e0-ab11-6e4bace72bc4 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:06.029713 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 700/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:08.009636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb9bd8-bd7b-429f-ba7a-8ea4a68430cd None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:08.010266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb9bd8-bd7b-429f-ba7a-8ea4a68430cd None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:08.010266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb9bd8-bd7b-429f-ba7a-8ea4a68430cd None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:08.011531 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 686/1386] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:08.017192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe956ef-149c-4f0a-a762-b5c6086dd727 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:08.017621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe956ef-149c-4f0a-a762-b5c6086dd727 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:08.017923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe956ef-149c-4f0a-a762-b5c6086dd727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:08.097424 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9fe956ef-149c-4f0a-a762-b5c6086dd727 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:08.098612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9fe956ef-149c-4f0a-a762-b5c6086dd727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:08.208870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 701/1387] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:08.522757 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb9bd8-bd7b-429f-ba7a-8ea4a68430cd None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:08.524455 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-daeb9bd8-bd7b-429f-ba7a-8ea4a68430cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:08.621700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:08.707218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb9bd8-bd7b-429f-ba7a-8ea4a68430cd None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:08.858367 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a21e1f-b759-4cdf-87be-86922183053a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:08.858988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a21e1f-b759-4cdf-87be-86922183053a service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:08.860321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a21e1f-b759-4cdf-87be-86922183053a service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:08.860598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a21e1f-b759-4cdf-87be-86922183053a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:08.863237 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-16a21e1f-b759-4cdf-87be-86922183053a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:08.916894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:08.989574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a21e1f-b759-4cdf-87be-86922183053a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:08.991726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a21e1f-b759-4cdf-87be-86922183053a service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:09.013636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 687/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:11.215383 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dfab5-6dfc-42da-b199-02598539d175 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:11.215975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dfab5-6dfc-42da-b199-02598539d175 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:11.215975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dfab5-6dfc-42da-b199-02598539d175 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:11.217315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 702/1389] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:11.222464 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9a424-58c8-4d29-8344-b78687dfd6bf None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:11.222912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9a424-58c8-4d29-8344-b78687dfd6bf None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:11.223016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9a424-58c8-4d29-8344-b78687dfd6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:11.289454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-04f9a424-58c8-4d29-8344-b78687dfd6bf None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:11.290651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-04f9a424-58c8-4d29-8344-b78687dfd6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:11.388037 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 688/1390] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:11.514520 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26b2f7-1168-45c6-8d51-e94171a7a766 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:11.514756 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26b2f7-1168-45c6-8d51-e94171a7a766 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:11.515044 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26b2f7-1168-45c6-8d51-e94171a7a766 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:11.516552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 703/1391] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:11.522402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abe319-9d08-4121-8efa-49ef02781743 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:11.522691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abe319-9d08-4121-8efa-49ef02781743 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:11.522965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abe319-9d08-4121-8efa-49ef02781743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:11.539260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:11.590292 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-d3abe319-9d08-4121-8efa-49ef02781743 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:11.591513 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d3abe319-9d08-4121-8efa-49ef02781743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:11.677491 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26b2f7-1168-45c6-8d51-e94171a7a766 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:11.679286 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-bc26b2f7-1168-45c6-8d51-e94171a7a766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:11.692150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 689/1392] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:11.711259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:11.825701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26b2f7-1168-45c6-8d51-e94171a7a766 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:11.927526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:11.939104 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee292b-3706-4207-ba4d-1c76da1f84a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:11.940010 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee292b-3706-4207-ba4d-1c76da1f84a7 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:11.940010 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee292b-3706-4207-ba4d-1c76da1f84a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:11.940218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee292b-3706-4207-ba4d-1c76da1f84a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:11.941874 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-efee292b-3706-4207-ba4d-1c76da1f84a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:11.994715 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfb6bf-2e70-4d38-9bb5-bca465d1edce None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:11.995007 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfb6bf-2e70-4d38-9bb5-bca465d1edce None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:11.995285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfb6bf-2e70-4d38-9bb5-bca465d1edce None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:11.996814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 690/1393] 10.222.0.27 () {54 vars in 967 bytes} [Fri Aug 12 12:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:12:12.001367 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ca503-46ee-41ec-8139-101e2b51ed91 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:12.001654 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ca503-46ee-41ec-8139-101e2b51ed91 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:12.001940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ca503-46ee-41ec-8139-101e2b51ed91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:12.033136 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee292b-3706-4207-ba4d-1c76da1f84a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:12.034742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee292b-3706-4207-ba4d-1c76da1f84a7 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:12.052827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 704/1394] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:12:12.077560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-6a7ca503-46ee-41ec-8139-101e2b51ed91 None None] MFA Rules not processed for user `ffe9f3fec7104a289e05e88e2d751458`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:12.078716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6a7ca503-46ee-41ec-8139-101e2b51ed91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:12.113671 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:12.155849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 691/1395] 10.222.0.27 () {58 vars in 1064 bytes} [Fri Aug 12 12:12:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:12:12.164672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee292b-3706-4207-ba4d-1c76da1f84a7 service nova] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:12.165727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-efee292b-3706-4207-ba4d-1c76da1f84a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:12.242157 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee292b-3706-4207-ba4d-1c76da1f84a7 service nova] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:12.289055 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:12.324998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03137207-3710-4480-8d22-f8cb4c90d770 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe9f3fec7104a289e05e88e2d751458', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:12.325608 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03137207-3710-4480-8d22-f8cb4c90d770 service glance] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:12.325891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03137207-3710-4480-8d22-f8cb4c90d770 service glance] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:12.326157 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03137207-3710-4480-8d22-f8cb4c90d770 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:12.327873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-03137207-3710-4480-8d22-f8cb4c90d770 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:12.443920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03137207-3710-4480-8d22-f8cb4c90d770 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:12.445523 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03137207-3710-4480-8d22-f8cb4c90d770 service glance] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:12.464282 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 705/1396] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:13.314000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab1618-95fe-49e4-bec6-2e9095c886ef None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:13.314574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab1618-95fe-49e4-bec6-2e9095c886ef None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:13.314574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab1618-95fe-49e4-bec6-2e9095c886ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:13.383772 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-3fab1618-95fe-49e4-bec6-2e9095c886ef None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:13.385144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-3fab1618-95fe-49e4-bec6-2e9095c886ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:13.474160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 692/1397] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Aug 12 12:12:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:13.522434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fc5fd-cc2a-46fa-aa39-0f398615c41c None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:13.522894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fc5fd-cc2a-46fa-aa39-0f398615c41c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:13.523007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fc5fd-cc2a-46fa-aa39-0f398615c41c None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:13.524322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 706/1398] 10.222.0.27 () {54 vars in 943 bytes} [Fri Aug 12 12:12:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:13.530422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57159f-90b0-4797-adf4-437edc3d7f60 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:13.530784 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57159f-90b0-4797-adf4-437edc3d7f60 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:13.531130 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57159f-90b0-4797-adf4-437edc3d7f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:13.590816 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:13.609237 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-5b57159f-90b0-4797-adf4-437edc3d7f60 None None] MFA Rules not processed for user `5123401076a84ffcab49b00fee65d14d`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:13.610480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-5b57159f-90b0-4797-adf4-437edc3d7f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:13.718323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 693/1399] 10.222.0.27 () {58 vars in 1040 bytes} [Fri Aug 12 12:12:13 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:13.728279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fc5fd-cc2a-46fa-aa39-0f398615c41c None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:13.729852 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-478fc5fd-cc2a-46fa-aa39-0f398615c41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:13.810530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:13.872123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fc5fd-cc2a-46fa-aa39-0f398615c41c None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:13.980101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8190d-168e-4d3f-a1b7-5b061d1b6307 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:13.980736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8190d-168e-4d3f-a1b7-5b061d1b6307 service swift] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:13.981019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8190d-168e-4d3f-a1b7-5b061d1b6307 service swift] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:13.981292 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8190d-168e-4d3f-a1b7-5b061d1b6307 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:13.982916 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b0f8190d-168e-4d3f-a1b7-5b061d1b6307 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:14.042068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:14.066769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8190d-168e-4d3f-a1b7-5b061d1b6307 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:14.068494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8190d-168e-4d3f-a1b7-5b061d1b6307 service swift] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:14.076314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 707/1400] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:12:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 12:12:14.158359 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9e852-573d-4a52-af3f-7213d591d084 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:14.158650 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9e852-573d-4a52-af3f-7213d591d084 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:14.158931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9e852-573d-4a52-af3f-7213d591d084 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:14.160282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 694/1401] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:12:14.166163 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a3a30-6d32-42e2-bde5-5e091ee8cb2e None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:14.166450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a3a30-6d32-42e2-bde5-5e091ee8cb2e None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:14.168551 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a3a30-6d32-42e2-bde5-5e091ee8cb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:14.233478 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-618a3a30-6d32-42e2-bde5-5e091ee8cb2e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:14.234690 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-618a3a30-6d32-42e2-bde5-5e091ee8cb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:14.332983 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 708/1402] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:14.658499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9e852-573d-4a52-af3f-7213d591d084 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:14.660145 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a3f9e852-573d-4a52-af3f-7213d591d084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:14.829891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:14.857105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9e852-573d-4a52-af3f-7213d591d084 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:15.014525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa38243-c81b-47ae-9ae9-6d447514bb9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:15.015441 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa38243-c81b-47ae-9ae9-6d447514bb9f service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:15.015914 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa38243-c81b-47ae-9ae9-6d447514bb9f service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:15.016194 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa38243-c81b-47ae-9ae9-6d447514bb9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:15.017901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-caa38243-c81b-47ae-9ae9-6d447514bb9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:15.209687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa38243-c81b-47ae-9ae9-6d447514bb9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:15.209687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa38243-c81b-47ae-9ae9-6d447514bb9f service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:15.209687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 695/1403] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:15.445872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:17.525118 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405cf8a-3801-4c07-8436-2acaee355b96 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:17.525617 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405cf8a-3801-4c07-8436-2acaee355b96 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:17.525617 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405cf8a-3801-4c07-8436-2acaee355b96 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:17.530780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 709/1404] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:12:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:12:17.532094 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9b8e8-351f-4c54-8e09-c4d5f3a8302f None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:17.532512 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9b8e8-351f-4c54-8e09-c4d5f3a8302f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:17.532718 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9b8e8-351f-4c54-8e09-c4d5f3a8302f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:17.610269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-4af9b8e8-351f-4c54-8e09-c4d5f3a8302f None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:17.611132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4af9b8e8-351f-4c54-8e09-c4d5f3a8302f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:17.719311 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 696/1405] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:12:18.012885 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405cf8a-3801-4c07-8436-2acaee355b96 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:18.014425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9405cf8a-3801-4c07-8436-2acaee355b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:18.069101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:18.171488 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405cf8a-3801-4c07-8436-2acaee355b96 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:18.305951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82573bb-1731-47b0-a936-f4eecf3fe00a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:18.305951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82573bb-1731-47b0-a936-f4eecf3fe00a service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:18.305951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82573bb-1731-47b0-a936-f4eecf3fe00a service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:18.306353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82573bb-1731-47b0-a936-f4eecf3fe00a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:18.307903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f82573bb-1731-47b0-a936-f4eecf3fe00a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:18.320814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:18.411302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82573bb-1731-47b0-a936-f4eecf3fe00a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:18.413398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82573bb-1731-47b0-a936-f4eecf3fe00a service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:18.433776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 710/1406] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:20.547447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fee0d5-e905-455d-bc86-39c547ffb143 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:20.547985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fee0d5-e905-455d-bc86-39c547ffb143 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:20.547985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fee0d5-e905-455d-bc86-39c547ffb143 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:20.549382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 697/1407] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:20.555247 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd487011-015b-4c54-b3a9-c92d211642f2 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:20.555727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd487011-015b-4c54-b3a9-c92d211642f2 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:20.555806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd487011-015b-4c54-b3a9-c92d211642f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:20.628444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-fd487011-015b-4c54-b3a9-c92d211642f2 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:20.629735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-fd487011-015b-4c54-b3a9-c92d211642f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:20.724885 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:20.738199 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 711/1408] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:21.043874 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fee0d5-e905-455d-bc86-39c547ffb143 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:21.045678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-23fee0d5-e905-455d-bc86-39c547ffb143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:21.108357 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:21.216283 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fee0d5-e905-455d-bc86-39c547ffb143 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:21.344289 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0aae57-1e9d-4c98-acdc-dc1decd5291c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:21.344847 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0aae57-1e9d-4c98-acdc-dc1decd5291c service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:21.345135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0aae57-1e9d-4c98-acdc-dc1decd5291c service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:21.346037 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0aae57-1e9d-4c98-acdc-dc1decd5291c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:21.347816 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-cf0aae57-1e9d-4c98-acdc-dc1decd5291c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:21.358077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:21.457099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0aae57-1e9d-4c98-acdc-dc1decd5291c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:21.459273 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0aae57-1e9d-4c98-acdc-dc1decd5291c service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:21.479139 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 698/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:30.786177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:31.476418 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=41 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:37.086553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041ad18-6911-4c33-a832-175542b3d5f3 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:37.087099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041ad18-6911-4c33-a832-175542b3d5f3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:37.087099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041ad18-6911-4c33-a832-175542b3d5f3 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:37.088458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 712/1410] 10.222.0.27 () {54 vars in 918 bytes} [Fri Aug 12 12:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:37.093501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b13c6-da6e-4fef-9d44-0f519a3f7299 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:37.093945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b13c6-da6e-4fef-9d44-0f519a3f7299 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:37.094060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b13c6-da6e-4fef-9d44-0f519a3f7299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:37.169246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-f40b13c6-da6e-4fef-9d44-0f519a3f7299 None None] MFA Rules not processed for user `5ee8ba21bc184feeaafacab3e69e87bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:37.170452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f40b13c6-da6e-4fef-9d44-0f519a3f7299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:37.275817 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 699/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Fri Aug 12 12:12:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:37.291135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040727f3-7b2c-4a54-92e1-b46782d4ada8 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:37.291416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040727f3-7b2c-4a54-92e1-b46782d4ada8 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:37.291709 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040727f3-7b2c-4a54-92e1-b46782d4ada8 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:37.293067 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 713/1412] 10.222.0.27 () {54 vars in 918 bytes} [Fri Aug 12 12:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:37.297914 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0939ba0-c521-4192-b671-29efa84f30d5 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:37.298199 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0939ba0-c521-4192-b671-29efa84f30d5 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:37.298472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0939ba0-c521-4192-b671-29efa84f30d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:37.370921 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:37.373851 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-d0939ba0-c521-4192-b671-29efa84f30d5 None None] MFA Rules not processed for user `687d0375abd44d2481f988c6df90fad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:37.375071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d0939ba0-c521-4192-b671-29efa84f30d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:37.490212 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 700/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Fri Aug 12 12:12:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:39.851905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722da47-7596-4667-ae36-c90a00f50acf None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:39.852421 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722da47-7596-4667-ae36-c90a00f50acf None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:39.852545 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722da47-7596-4667-ae36-c90a00f50acf None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:39.858209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 12:12:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:12:39.864372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd83f8c-6ab4-4c7e-baf3-e627a7fac827 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:39.864832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd83f8c-6ab4-4c7e-baf3-e627a7fac827 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:39.864988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd83f8c-6ab4-4c7e-baf3-e627a7fac827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:39.930250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-4cd83f8c-6ab4-4c7e-baf3-e627a7fac827 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:39.931464 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4cd83f8c-6ab4-4c7e-baf3-e627a7fac827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:39.985658 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:40.027946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 12:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:40.321357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722da47-7596-4667-ae36-c90a00f50acf None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:40.322903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4722da47-7596-4667-ae36-c90a00f50acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:40.468007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722da47-7596-4667-ae36-c90a00f50acf None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:40.538979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:40.579079 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86cb656-067e-41eb-b685-6a8ca897c87e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:40.579608 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cb656-067e-41eb-b685-6a8ca897c87e service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:40.579890 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cb656-067e-41eb-b685-6a8ca897c87e service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:40.580225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cb656-067e-41eb-b685-6a8ca897c87e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:40.582499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e86cb656-067e-41eb-b685-6a8ca897c87e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:40.670763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86cb656-067e-41eb-b685-6a8ca897c87e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:40.672351 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86cb656-067e-41eb-b685-6a8ca897c87e service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:40.690536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 715/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:41.593484 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3005d-8d56-4d83-847b-3f6159c46874 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:41.594365 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3005d-8d56-4d83-847b-3f6159c46874 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:41.594635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3005d-8d56-4d83-847b-3f6159c46874 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:41.596010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 12:12:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:41.601014 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9102d-0de6-4071-a0d5-0cd97b4d58b6 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:41.601434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9102d-0de6-4071-a0d5-0cd97b4d58b6 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:41.601563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9102d-0de6-4071-a0d5-0cd97b4d58b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:41.675130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-abe9102d-0de6-4071-a0d5-0cd97b4d58b6 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:41.676838 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-abe9102d-0de6-4071-a0d5-0cd97b4d58b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:41.703492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:41.772120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 12:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:41.923884 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3005d-8d56-4d83-847b-3f6159c46874 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:41.925557 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-aac3005d-8d56-4d83-847b-3f6159c46874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:42.069828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3005d-8d56-4d83-847b-3f6159c46874 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:42.109014 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:42.180712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6f551-a0c0-4de3-8ffb-c68c78820aaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:42.181251 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6f551-a0c0-4de3-8ffb-c68c78820aaf service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:42.181535 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6f551-a0c0-4de3-8ffb-c68c78820aaf service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:42.181809 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6f551-a0c0-4de3-8ffb-c68c78820aaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:42.183410 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a1a6f551-a0c0-4de3-8ffb-c68c78820aaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:42.271672 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a6f551-a0c0-4de3-8ffb-c68c78820aaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:42.273288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a6f551-a0c0-4de3-8ffb-c68c78820aaf service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:42.290939 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 703/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:43.173142 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3374eb7-f1e2-4943-8a8a-7602cc7c7150 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:43.173709 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3374eb7-f1e2-4943-8a8a-7602cc7c7150 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:43.173709 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3374eb7-f1e2-4943-8a8a-7602cc7c7150 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:43.175056 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 12:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:43.180346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0875e-5fdb-42a5-9023-c105b2929204 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:43.180776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0875e-5fdb-42a5-9023-c105b2929204 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:43.180956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0875e-5fdb-42a5-9023-c105b2929204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:43.234237 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:43.255641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-5ef0875e-5fdb-42a5-9023-c105b2929204 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:43.256878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-5ef0875e-5fdb-42a5-9023-c105b2929204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:43.354097 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 12:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:43.508730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3374eb7-f1e2-4943-8a8a-7602cc7c7150 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:43.510180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-d3374eb7-f1e2-4943-8a8a-7602cc7c7150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:43.653854 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3374eb7-f1e2-4943-8a8a-7602cc7c7150 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:43.665926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:43.765904 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4bb7e-1db1-4d48-8130-9eff30b1efc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:43.766468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4bb7e-1db1-4d48-8130-9eff30b1efc7 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:43.766779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4bb7e-1db1-4d48-8130-9eff30b1efc7 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:43.767052 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4bb7e-1db1-4d48-8130-9eff30b1efc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:43.768664 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-edf4bb7e-1db1-4d48-8130-9eff30b1efc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:43.856838 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4bb7e-1db1-4d48-8130-9eff30b1efc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:43.858420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4bb7e-1db1-4d48-8130-9eff30b1efc7 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:43.876175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 718/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:44.737652 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8094088-b469-4d2e-bd6d-b4ed9fac3857 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:44.738224 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8094088-b469-4d2e-bd6d-b4ed9fac3857 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:44.738332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8094088-b469-4d2e-bd6d-b4ed9fac3857 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:44.739709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 12:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:44.744904 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88877a87-c6d9-477c-b10d-107d8c0a7d98 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:44.745361 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88877a87-c6d9-477c-b10d-107d8c0a7d98 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:44.745469 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88877a87-c6d9-477c-b10d-107d8c0a7d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:44.764988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:44.822829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-88877a87-c6d9-477c-b10d-107d8c0a7d98 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:44.824024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-88877a87-c6d9-477c-b10d-107d8c0a7d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:44.917563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 12:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:45.066992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8094088-b469-4d2e-bd6d-b4ed9fac3857 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:45.068599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c8094088-b469-4d2e-bd6d-b4ed9fac3857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:45.176554 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:45.212947 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8094088-b469-4d2e-bd6d-b4ed9fac3857 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:45.323678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62269b-27f7-4f93-8d4f-9ffed921912e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:45.324199 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62269b-27f7-4f93-8d4f-9ffed921912e service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:45.324479 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62269b-27f7-4f93-8d4f-9ffed921912e service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:45.324788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62269b-27f7-4f93-8d4f-9ffed921912e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:45.326389 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0e62269b-27f7-4f93-8d4f-9ffed921912e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:45.402869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:45.416141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62269b-27f7-4f93-8d4f-9ffed921912e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:45.417761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62269b-27f7-4f93-8d4f-9ffed921912e service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:45.438281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 706/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:46.298797 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03095f68-97a2-450f-be2c-bc1e99e13dfc None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:46.299335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03095f68-97a2-450f-be2c-bc1e99e13dfc None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:46.299335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03095f68-97a2-450f-be2c-bc1e99e13dfc None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:46.300823 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 12:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:46.306039 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ab2e2-0995-45a9-ae80-69f9a07881c6 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:46.306446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ab2e2-0995-45a9-ae80-69f9a07881c6 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:46.306593 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ab2e2-0995-45a9-ae80-69f9a07881c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:46.382003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-215ab2e2-0995-45a9-ae80-69f9a07881c6 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:46.383223 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-215ab2e2-0995-45a9-ae80-69f9a07881c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:46.483132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 12:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:46.642887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03095f68-97a2-450f-be2c-bc1e99e13dfc None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:46.644969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-03095f68-97a2-450f-be2c-bc1e99e13dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:46.727580 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:46.788770 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03095f68-97a2-450f-be2c-bc1e99e13dfc None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:46.900904 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c6e14-39c1-421c-899f-801981d436e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:46.901517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c6e14-39c1-421c-899f-801981d436e5 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:46.901845 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c6e14-39c1-421c-899f-801981d436e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:46.902119 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c6e14-39c1-421c-899f-801981d436e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:46.903782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-775c6e14-39c1-421c-899f-801981d436e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:46.951071 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:46.994351 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c6e14-39c1-421c-899f-801981d436e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:46.995958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c6e14-39c1-421c-899f-801981d436e5 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:47.013773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 721/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:47.910142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e7411-8241-458f-b5e5-3703a8f63e3a None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:47.910690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e7411-8241-458f-b5e5-3703a8f63e3a None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:47.910690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e7411-8241-458f-b5e5-3703a8f63e3a None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:47.912108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 12:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:47.917105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5157d6-57ba-47c8-b866-0e9432947f52 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:47.917528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5157d6-57ba-47c8-b866-0e9432947f52 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:47.917668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5157d6-57ba-47c8-b866-0e9432947f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:48.111795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-ec5157d6-57ba-47c8-b866-0e9432947f52 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:48.113084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ec5157d6-57ba-47c8-b866-0e9432947f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:48.209122 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 12:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:48.358132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e7411-8241-458f-b5e5-3703a8f63e3a None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:48.359642 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-da7e7411-8241-458f-b5e5-3703a8f63e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:48.410149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:48.510603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e7411-8241-458f-b5e5-3703a8f63e3a None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:48.623823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba26e7-ec71-4682-a435-eb92e7f78055 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:48.624426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba26e7-ec71-4682-a435-eb92e7f78055 service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:48.624743 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba26e7-ec71-4682-a435-eb92e7f78055 service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:48.625020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba26e7-ec71-4682-a435-eb92e7f78055 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:48.626628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b7ba26e7-ec71-4682-a435-eb92e7f78055 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:48.633703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:48.714641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba26e7-ec71-4682-a435-eb92e7f78055 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:48.716157 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba26e7-ec71-4682-a435-eb92e7f78055 service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:48.734241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 709/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:49.598396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ac239-537d-4198-a6ea-b4dbcd484a44 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:49.598934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ac239-537d-4198-a6ea-b4dbcd484a44 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:49.598934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ac239-537d-4198-a6ea-b4dbcd484a44 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:49.600298 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 12:12:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:49.605363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667274b-a26d-489c-8727-b4fa7c959ab7 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:49.605798 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667274b-a26d-489c-8727-b4fa7c959ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:49.605909 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667274b-a26d-489c-8727-b4fa7c959ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:49.680481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-9667274b-a26d-489c-8727-b4fa7c959ab7 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:49.681698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9667274b-a26d-489c-8727-b4fa7c959ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:49.749053 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:49.777770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 12:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:49.932779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ac239-537d-4198-a6ea-b4dbcd484a44 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:49.934339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e37ac239-537d-4198-a6ea-b4dbcd484a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:49.952782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:50.082198 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ac239-537d-4198-a6ea-b4dbcd484a44 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:50.167527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:50.192944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fbe97-55dc-4663-b9b6-ac588efbdb15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:50.193481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fbe97-55dc-4663-b9b6-ac588efbdb15 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:50.193762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fbe97-55dc-4663-b9b6-ac588efbdb15 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:50.194033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fbe97-55dc-4663-b9b6-ac588efbdb15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:50.195627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-3f5fbe97-55dc-4663-b9b6-ac588efbdb15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:50.289017 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5fbe97-55dc-4663-b9b6-ac588efbdb15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:50.290623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5fbe97-55dc-4663-b9b6-ac588efbdb15 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:50.309290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 724/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:51.168807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c463d90-e0ca-4d65-bcaf-eee86c9f7959 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:51.169332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c463d90-e0ca-4d65-bcaf-eee86c9f7959 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:51.169332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c463d90-e0ca-4d65-bcaf-eee86c9f7959 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:51.170699 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 12:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:51.176236 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78262b9b-5a4a-4279-9489-8a3818b064d2 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:51.176660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78262b9b-5a4a-4279-9489-8a3818b064d2 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:51.176827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78262b9b-5a4a-4279-9489-8a3818b064d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:51.252757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-78262b9b-5a4a-4279-9489-8a3818b064d2 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:51.254706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-78262b9b-5a4a-4279-9489-8a3818b064d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:51.295116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:51.351035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 12:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:51.502494 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c463d90-e0ca-4d65-bcaf-eee86c9f7959 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:51.504709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1c463d90-e0ca-4d65-bcaf-eee86c9f7959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:51.646337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c463d90-e0ca-4d65-bcaf-eee86c9f7959 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:51.699340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:51.758263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a97015-26ed-46a3-a2c1-049c463155fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:51.758813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a97015-26ed-46a3-a2c1-049c463155fc service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:51.759099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a97015-26ed-46a3-a2c1-049c463155fc service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:51.759378 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a97015-26ed-46a3-a2c1-049c463155fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:51.761050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d8a97015-26ed-46a3-a2c1-049c463155fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:51.850652 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a97015-26ed-46a3-a2c1-049c463155fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:51.852224 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a97015-26ed-46a3-a2c1-049c463155fc service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:51.870099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 712/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:52.715448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfd481-bc74-45bf-925b-61b678d73598 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:52.715942 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfd481-bc74-45bf-925b-61b678d73598 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:52.716036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfd481-bc74-45bf-925b-61b678d73598 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:52.717355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 12:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:52.722431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf8e6c-e73e-42c7-a972-9d94b137b4e9 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:52.722886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf8e6c-e73e-42c7-a972-9d94b137b4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:52.723085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf8e6c-e73e-42c7-a972-9d94b137b4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:52.798527 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-6fdf8e6c-e73e-42c7-a972-9d94b137b4e9 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:52.800229 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6fdf8e6c-e73e-42c7-a972-9d94b137b4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:52.802428 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:53.016667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 12:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:53.167742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfd481-bc74-45bf-925b-61b678d73598 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:53.169388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4edfd481-bc74-45bf-925b-61b678d73598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:53.311312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfd481-bc74-45bf-925b-61b678d73598 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:53.338948 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:53.426498 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdbcbb-3640-42bc-a081-875def3fbea9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:53.427109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdbcbb-3640-42bc-a081-875def3fbea9 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:53.427370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdbcbb-3640-42bc-a081-875def3fbea9 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:53.427639 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdbcbb-3640-42bc-a081-875def3fbea9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:53.429534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-aecdbcbb-3640-42bc-a081-875def3fbea9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:53.517933 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdbcbb-3640-42bc-a081-875def3fbea9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:53.519683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdbcbb-3640-42bc-a081-875def3fbea9 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:53.537274 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 727/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:55.345069 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4607c-f05b-4603-ab08-1f72a5429a3d None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:55.345611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4607c-f05b-4603-ab08-1f72a5429a3d None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:55.345611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4607c-f05b-4603-ab08-1f72a5429a3d None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:55.347735 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 12:12:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:55.353199 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0463a-1d3f-4dfd-9778-a28f3adf1e3f None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:55.353629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0463a-1d3f-4dfd-9778-a28f3adf1e3f None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:55.353744 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0463a-1d3f-4dfd-9778-a28f3adf1e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:55.394305 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:55.419269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-a7b0463a-1d3f-4dfd-9778-a28f3adf1e3f None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:55.420475 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a7b0463a-1d3f-4dfd-9778-a28f3adf1e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:55.525583 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 12:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:12:55.810468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4607c-f05b-4603-ab08-1f72a5429a3d None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:55.812000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b8b4607c-f05b-4603-ab08-1f72a5429a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:55.915132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4607c-f05b-4603-ab08-1f72a5429a3d None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:55.946278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:55.987478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b930b55-a18b-4979-8e1d-bac1d5eecb50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:55.987975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b930b55-a18b-4979-8e1d-bac1d5eecb50 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:55.988254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b930b55-a18b-4979-8e1d-bac1d5eecb50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:55.988522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b930b55-a18b-4979-8e1d-bac1d5eecb50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:55.990145 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9b930b55-a18b-4979-8e1d-bac1d5eecb50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:56.096083 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b930b55-a18b-4979-8e1d-bac1d5eecb50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:56.098064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b930b55-a18b-4979-8e1d-bac1d5eecb50 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:56.116073 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 715/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:12:58.158862 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e5a07-cd10-4ba8-807c-531ca81671ea None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:58.159436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e5a07-cd10-4ba8-807c-531ca81671ea None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:58.159436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e5a07-cd10-4ba8-807c-531ca81671ea None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:58.160770 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 12:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:12:58.165941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107a3d3-f84a-41a9-982c-0c64211dfe05 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:58.166356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107a3d3-f84a-41a9-982c-0c64211dfe05 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:58.166497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107a3d3-f84a-41a9-982c-0c64211dfe05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:58.231664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-1107a3d3-f84a-41a9-982c-0c64211dfe05 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:12:58.232895 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1107a3d3-f84a-41a9-982c-0c64211dfe05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:58.245898 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:58.329853 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 12:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:12:58.617955 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e5a07-cd10-4ba8-807c-531ca81671ea None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:12:58.619485 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4c5e5a07-cd10-4ba8-807c-531ca81671ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:58.721928 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e5a07-cd10-4ba8-807c-531ca81671ea None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:12:58.734002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:12:58.794881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9443b6e-5b33-453d-ad32-935fec985b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:12:58.795486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9443b6e-5b33-453d-ad32-935fec985b7d service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:12:58.795770 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9443b6e-5b33-453d-ad32-935fec985b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:12:58.796047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9443b6e-5b33-453d-ad32-935fec985b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:12:58.797716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b9443b6e-5b33-453d-ad32-935fec985b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:12:58.907021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9443b6e-5b33-453d-ad32-935fec985b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:12:58.908479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9443b6e-5b33-453d-ad32-935fec985b7d service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:12:58.928443 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 730/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:12:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:13:01.166916 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c1f3e2-6d16-4e05-aaca-56066b6ef309 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:01.167457 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c1f3e2-6d16-4e05-aaca-56066b6ef309 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:01.167457 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c1f3e2-6d16-4e05-aaca-56066b6ef309 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:01.168884 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 12:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:01.174013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d1053-711c-447f-8085-ec0c59bac141 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:01.174425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d1053-711c-447f-8085-ec0c59bac141 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:01.174681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d1053-711c-447f-8085-ec0c59bac141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:01.222911 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:01.240540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-244d1053-711c-447f-8085-ec0c59bac141 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:01.241780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-244d1053-711c-447f-8085-ec0c59bac141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:01.335725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 12:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:13:01.620032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c1f3e2-6d16-4e05-aaca-56066b6ef309 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:13:01.621601 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-31c1f3e2-6d16-4e05-aaca-56066b6ef309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:01.727026 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c1f3e2-6d16-4e05-aaca-56066b6ef309 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:13:01.768852 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:01.803315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86ffe0-19f3-4de0-be1a-85d0b42c0a51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:13:01.803916 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86ffe0-19f3-4de0-be1a-85d0b42c0a51 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:01.804197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86ffe0-19f3-4de0-be1a-85d0b42c0a51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:01.804470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86ffe0-19f3-4de0-be1a-85d0b42c0a51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:01.806153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0e86ffe0-19f3-4de0-be1a-85d0b42c0a51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:01.912874 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86ffe0-19f3-4de0-be1a-85d0b42c0a51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:13:01.914298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86ffe0-19f3-4de0-be1a-85d0b42c0a51 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:13:01.932534 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 718/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:13:04.012697 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5e4fc-bf0b-4f30-b062-a20a8270110b None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:04.013255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5e4fc-bf0b-4f30-b062-a20a8270110b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:04.013255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5e4fc-bf0b-4f30-b062-a20a8270110b None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:04.014572 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 12:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:04.020281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2b72b-0ca2-4ce2-90b7-7da4973f9d18 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:04.020725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2b72b-0ca2-4ce2-90b7-7da4973f9d18 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:04.020850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2b72b-0ca2-4ce2-90b7-7da4973f9d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:04.087275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-9fc2b72b-0ca2-4ce2-90b7-7da4973f9d18 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:04.088423 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9fc2b72b-0ca2-4ce2-90b7-7da4973f9d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:04.109345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:04.183887 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 12:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:13:04.467925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5e4fc-bf0b-4f30-b062-a20a8270110b None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:13:04.469498 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0ae5e4fc-bf0b-4f30-b062-a20a8270110b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:04.571416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5e4fc-bf0b-4f30-b062-a20a8270110b None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:13:04.590772 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:04.645076 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58693790-c27e-4560-8462-692108eed644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:13:04.645674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58693790-c27e-4560-8462-692108eed644 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:04.645953 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58693790-c27e-4560-8462-692108eed644 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:04.646221 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58693790-c27e-4560-8462-692108eed644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:04.647811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-58693790-c27e-4560-8462-692108eed644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:04.756094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58693790-c27e-4560-8462-692108eed644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:13:04.757456 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58693790-c27e-4560-8462-692108eed644 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:13:04.775668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 733/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:13:06.730839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b29a2b-b5cf-485f-92f0-df1685c52689 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:06.731440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b29a2b-b5cf-485f-92f0-df1685c52689 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:06.731440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b29a2b-b5cf-485f-92f0-df1685c52689 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:06.732799 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 12:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:06.738775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392d94b-62f0-408c-bcd6-b32c379e07a4 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:06.739222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392d94b-62f0-408c-bcd6-b32c379e07a4 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:06.739323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392d94b-62f0-408c-bcd6-b32c379e07a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:06.805637 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:06.808452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-b392d94b-62f0-408c-bcd6-b32c379e07a4 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:06.810030 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b392d94b-62f0-408c-bcd6-b32c379e07a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:06.904359 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 12:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:13:07.191889 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b29a2b-b5cf-485f-92f0-df1685c52689 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:13:07.193471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-06b29a2b-b5cf-485f-92f0-df1685c52689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:07.299353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b29a2b-b5cf-485f-92f0-df1685c52689 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:13:07.348872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:07.372631 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beaeec7-7150-43c0-adb2-03ad51efdc5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:13:07.373250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaeec7-7150-43c0-adb2-03ad51efdc5e service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:07.373531 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaeec7-7150-43c0-adb2-03ad51efdc5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:07.373799 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaeec7-7150-43c0-adb2-03ad51efdc5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:07.375380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9beaeec7-7150-43c0-adb2-03ad51efdc5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:07.483502 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beaeec7-7150-43c0-adb2-03ad51efdc5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:13:07.484866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beaeec7-7150-43c0-adb2-03ad51efdc5e service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:13:07.503601 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 721/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:13:09.525088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ef95b-825c-47ff-b4bc-2bafa6d87ac0 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:09.525787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ef95b-825c-47ff-b4bc-2bafa6d87ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:09.526014 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ef95b-825c-47ff-b4bc-2bafa6d87ac0 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:09.528026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 12:13:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:09.534275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa13f5-c88e-4bb8-820d-88f51dcacecb None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:09.534732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa13f5-c88e-4bb8-820d-88f51dcacecb None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:09.534834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa13f5-c88e-4bb8-820d-88f51dcacecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:09.599863 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-edfa13f5-c88e-4bb8-820d-88f51dcacecb None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:09.601103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-edfa13f5-c88e-4bb8-820d-88f51dcacecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:09.630092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:09.697062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 12:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:13:09.979279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ef95b-825c-47ff-b4bc-2bafa6d87ac0 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:13:09.980896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-229ef95b-825c-47ff-b4bc-2bafa6d87ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:10.083062 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ef95b-825c-47ff-b4bc-2bafa6d87ac0 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:13:10.112117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:10.160565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a99536-f01b-468c-acc6-6f01d4d6e2a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:13:10.161193 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a99536-f01b-468c-acc6-6f01d4d6e2a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:10.161472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a99536-f01b-468c-acc6-6f01d4d6e2a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:10.161744 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a99536-f01b-468c-acc6-6f01d4d6e2a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:10.163355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a7a99536-f01b-468c-acc6-6f01d4d6e2a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:10.275885 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a99536-f01b-468c-acc6-6f01d4d6e2a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:13:10.277363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a99536-f01b-468c-acc6-6f01d4d6e2a9 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:13:10.295802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 736/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:13:12.348725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89897b69-2ca1-47db-a524-cbd5f3412039 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:12.349266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89897b69-2ca1-47db-a524-cbd5f3412039 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:12.349266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89897b69-2ca1-47db-a524-cbd5f3412039 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:12.350638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 12:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:12.356042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1d05b-7b5f-4936-8d49-b04fe4dd2882 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:12.356485 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1d05b-7b5f-4936-8d49-b04fe4dd2882 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:12.356591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1d05b-7b5f-4936-8d49-b04fe4dd2882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:12.422786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f2e1d05b-7b5f-4936-8d49-b04fe4dd2882 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:12.424584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f2e1d05b-7b5f-4936-8d49-b04fe4dd2882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:12.433201 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:12.522374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 12:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:13:12.809617 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89897b69-2ca1-47db-a524-cbd5f3412039 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:13:12.812920 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-89897b69-2ca1-47db-a524-cbd5f3412039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:12.916239 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89897b69-2ca1-47db-a524-cbd5f3412039 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:13:12.973789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:12.992805 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a9fda-8d0a-4c1c-9809-c4a54fd17b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:13:12.993404 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a9fda-8d0a-4c1c-9809-c4a54fd17b4a service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:12.993686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a9fda-8d0a-4c1c-9809-c4a54fd17b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:12.993958 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a9fda-8d0a-4c1c-9809-c4a54fd17b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:12.995559 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-ca9a9fda-8d0a-4c1c-9809-c4a54fd17b4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:13.101866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a9fda-8d0a-4c1c-9809-c4a54fd17b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:13:13.103407 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a9fda-8d0a-4c1c-9809-c4a54fd17b4a service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:13:13.122241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 724/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:13:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:13:15.057909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03433a78-5b41-4be3-b860-71a7c39747f9 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:15.058464 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03433a78-5b41-4be3-b860-71a7c39747f9 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:15.058464 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03433a78-5b41-4be3-b860-71a7c39747f9 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:15.059977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 12:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:15.065414 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e75f3a-8ee2-45e5-bb3b-be527dbbd00b None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:15.065884 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e75f3a-8ee2-45e5-bb3b-be527dbbd00b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:15.065981 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e75f3a-8ee2-45e5-bb3b-be527dbbd00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:15.132659 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-88e75f3a-8ee2-45e5-bb3b-be527dbbd00b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:15.133922 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-88e75f3a-8ee2-45e5-bb3b-be527dbbd00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:15.172811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:15.229214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 12:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:13:15.512878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03433a78-5b41-4be3-b860-71a7c39747f9 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:13:15.514430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-03433a78-5b41-4be3-b860-71a7c39747f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:15.617638 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03433a78-5b41-4be3-b860-71a7c39747f9 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:13:15.654098 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:15.689992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16802d85-0064-4d08-837c-b2c912ae43ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:13:15.690592 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16802d85-0064-4d08-837c-b2c912ae43ff service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:15.690871 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16802d85-0064-4d08-837c-b2c912ae43ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:15.691140 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16802d85-0064-4d08-837c-b2c912ae43ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:15.693456 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-16802d85-0064-4d08-837c-b2c912ae43ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:15.800197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16802d85-0064-4d08-837c-b2c912ae43ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:13:15.801581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16802d85-0064-4d08-837c-b2c912ae43ff service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:13:15.820566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 739/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:13:17.727028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189363dc-a054-48f2-b987-3f9a8a2e6a52 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189363dc-a054-48f2-b987-3f9a8a2e6a52 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:17.727558 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189363dc-a054-48f2-b987-3f9a8a2e6a52 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:17.731582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 12:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:17.734113 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f7c4c-e723-4288-8287-4fb7ac41ce91 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:17.734539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f7c4c-e723-4288-8287-4fb7ac41ce91 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:17.734660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f7c4c-e723-4288-8287-4fb7ac41ce91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:17.800974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-143f7c4c-e723-4288-8287-4fb7ac41ce91 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:17.802175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-143f7c4c-e723-4288-8287-4fb7ac41ce91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:17.821195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:17.897887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 12:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:13:18.178559 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189363dc-a054-48f2-b987-3f9a8a2e6a52 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:13:18.180093 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-189363dc-a054-48f2-b987-3f9a8a2e6a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:18.280398 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189363dc-a054-48f2-b987-3f9a8a2e6a52 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:13:18.346234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:18.352748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273b6cf-f9e1-4b54-906d-d5071d0eaae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:13:18.353337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273b6cf-f9e1-4b54-906d-d5071d0eaae1 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:18.353614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273b6cf-f9e1-4b54-906d-d5071d0eaae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:18.353883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273b6cf-f9e1-4b54-906d-d5071d0eaae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:18.355444 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a273b6cf-f9e1-4b54-906d-d5071d0eaae1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:18.464586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273b6cf-f9e1-4b54-906d-d5071d0eaae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:13:18.466068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273b6cf-f9e1-4b54-906d-d5071d0eaae1 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:13:18.483862 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 727/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:13:20.397528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487bdba-9b8e-443c-8c2d-7e6051aa5845 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:20.398085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487bdba-9b8e-443c-8c2d-7e6051aa5845 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:20.398085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487bdba-9b8e-443c-8c2d-7e6051aa5845 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:20.399352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 12:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:20.404779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b9842-20a4-4f43-a048-138b51f9f432 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:20.405238 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b9842-20a4-4f43-a048-138b51f9f432 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:20.405342 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b9842-20a4-4f43-a048-138b51f9f432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:20.470861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-350b9842-20a4-4f43-a048-138b51f9f432 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:20.472063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-350b9842-20a4-4f43-a048-138b51f9f432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:20.520703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:20.569008 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 12:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:13:20.853754 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487bdba-9b8e-443c-8c2d-7e6051aa5845 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:13:20.855314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9487bdba-9b8e-443c-8c2d-7e6051aa5845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:20.958067 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487bdba-9b8e-443c-8c2d-7e6051aa5845 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:13:21.006464 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:21.032242 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e91fa-9860-4cee-87ba-f8d41964c0b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:13:21.032867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e91fa-9860-4cee-87ba-f8d41964c0b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:21.033662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e91fa-9860-4cee-87ba-f8d41964c0b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:21.034159 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e91fa-9860-4cee-87ba-f8d41964c0b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:21.035966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-3c7e91fa-9860-4cee-87ba-f8d41964c0b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:21.144458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e91fa-9860-4cee-87ba-f8d41964c0b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:13:21.145872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e91fa-9860-4cee-87ba-f8d41964c0b6 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:13:21.163738 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 742/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:13:30.567245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=22 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:31.161521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=66 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:40.725859 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24d8f7-1af5-455b-976b-b68e3cf0d2bb None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:40.732812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24d8f7-1af5-455b-976b-b68e3cf0d2bb None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:40.732812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24d8f7-1af5-455b-976b-b68e3cf0d2bb None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:40.732812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 729/1471] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:40.733099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1c747-e41d-4d29-84cf-e218891c4625 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:40.733484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1c747-e41d-4d29-84cf-e218891c4625 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:40.733484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1c747-e41d-4d29-84cf-e218891c4625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:40.798709 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-20d1c747-e41d-4d29-84cf-e218891c4625 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:40.799915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-20d1c747-e41d-4d29-84cf-e218891c4625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:40.894812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 743/1472] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:13:41.190932 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7d4f9-ca89-40e9-a0fb-0c4ff705411b None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:41.191218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7d4f9-ca89-40e9-a0fb-0c4ff705411b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:41.191489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7d4f9-ca89-40e9-a0fb-0c4ff705411b None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:41.192838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 730/1473] 10.222.0.27 () {54 vars in 967 bytes} [Fri Aug 12 12:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:41.197489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07686c-3da6-4eec-9b38-742bbfcf30ed None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:41.197851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07686c-3da6-4eec-9b38-742bbfcf30ed None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:41.198127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07686c-3da6-4eec-9b38-742bbfcf30ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:41.275974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-0a07686c-3da6-4eec-9b38-742bbfcf30ed None None] MFA Rules not processed for user `ffe9f3fec7104a289e05e88e2d751458`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:41.277198 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0a07686c-3da6-4eec-9b38-742bbfcf30ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:41.297277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:41.352776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 744/1474] 10.222.0.27 () {58 vars in 1064 bytes} [Fri Aug 12 12:13:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:13:41.360864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7d4f9-ca89-40e9-a0fb-0c4ff705411b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:13:41.362362 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-cca7d4f9-ca89-40e9-a0fb-0c4ff705411b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:41.464493 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7d4f9-ca89-40e9-a0fb-0c4ff705411b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:13:41.537910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd17e68-9460-47ef-bd14-d479bda561e5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe9f3fec7104a289e05e88e2d751458', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:13:41.538505 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd17e68-9460-47ef-bd14-d479bda561e5 service glance] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:41.538786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd17e68-9460-47ef-bd14-d479bda561e5 service glance] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:41.539056 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd17e68-9460-47ef-bd14-d479bda561e5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:41.540672 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1cd17e68-9460-47ef-bd14-d479bda561e5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:41.595213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:41.648937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd17e68-9460-47ef-bd14-d479bda561e5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:13:41.650339 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd17e68-9460-47ef-bd14-d479bda561e5 service glance] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:13:41.668031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 731/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:13:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:13:43.657678 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56795d8-6d02-43e8-b38a-d69e7ee1689c None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:43.658260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56795d8-6d02-43e8-b38a-d69e7ee1689c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:43.658260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56795d8-6d02-43e8-b38a-d69e7ee1689c None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:43.659587 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 745/1476] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:43.665204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae22165-0dcc-4937-8a30-74bd1541cf8a None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:43.665626 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae22165-0dcc-4937-8a30-74bd1541cf8a None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:43.665738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae22165-0dcc-4937-8a30-74bd1541cf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:43.730294 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-6ae22165-0dcc-4937-8a30-74bd1541cf8a None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:43.740823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6ae22165-0dcc-4937-8a30-74bd1541cf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:43.833098 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 732/1477] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:13:44.103445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56795d8-6d02-43e8-b38a-d69e7ee1689c None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:13:44.105014 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-c56795d8-6d02-43e8-b38a-d69e7ee1689c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:44.227752 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56795d8-6d02-43e8-b38a-d69e7ee1689c None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:13:44.275950 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:44.319585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63b08f-6796-4290-8cf5-cb2bb5ee6253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:13:44.320295 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63b08f-6796-4290-8cf5-cb2bb5ee6253 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:44.320630 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63b08f-6796-4290-8cf5-cb2bb5ee6253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:44.321109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63b08f-6796-4290-8cf5-cb2bb5ee6253 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:44.322181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63b08f-6796-4290-8cf5-cb2bb5ee6253 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:13:44.323923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: WARNING py.warnings [None req-5a63b08f-6796-4290-8cf5-cb2bb5ee6253 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 12:13:44.323923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: warnings.warn(msg) Aug 12 12:13:44.323923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: Aug 12 12:13:44.325110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63b08f-6796-4290-8cf5-cb2bb5ee6253 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:13:44.344790 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 746/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 12 12:13:44 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:13:46.242746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf2853-57be-4aa6-9a7f-1e53a07dfe49 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:46.243337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf2853-57be-4aa6-9a7f-1e53a07dfe49 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:46.243337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf2853-57be-4aa6-9a7f-1e53a07dfe49 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:46.244621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 733/1479] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:46.249692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370f267-5ecd-4171-a231-6e41dfcab019 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:46.250608 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370f267-5ecd-4171-a231-6e41dfcab019 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:46.250865 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370f267-5ecd-4171-a231-6e41dfcab019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:46.316441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f370f267-5ecd-4171-a231-6e41dfcab019 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:46.317655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f370f267-5ecd-4171-a231-6e41dfcab019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:46.416349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 747/1480] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:13:46.705387 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf2853-57be-4aa6-9a7f-1e53a07dfe49 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:13:46.706898 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-3fdf2853-57be-4aa6-9a7f-1e53a07dfe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:46.716523 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:46.849936 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf2853-57be-4aa6-9a7f-1e53a07dfe49 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:13:46.928542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:46.962018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b00553-bbe3-4526-809e-39ef29a9cf22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:13:46.962537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b00553-bbe3-4526-809e-39ef29a9cf22 service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:46.962814 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b00553-bbe3-4526-809e-39ef29a9cf22 service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:46.963080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b00553-bbe3-4526-809e-39ef29a9cf22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:46.964677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a4b00553-bbe3-4526-809e-39ef29a9cf22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:47.052698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b00553-bbe3-4526-809e-39ef29a9cf22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:13:47.054254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b00553-bbe3-4526-809e-39ef29a9cf22 service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:13:47.071992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 734/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:13:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:13:49.146745 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fcc97-f296-4263-8beb-ed75cf070b24 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:49.147253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fcc97-f296-4263-8beb-ed75cf070b24 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:49.147253 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fcc97-f296-4263-8beb-ed75cf070b24 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:49.148557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 748/1482] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:49.153877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cde886-4fbc-42a3-bc11-e59d98440ab6 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:49.154293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cde886-4fbc-42a3-bc11-e59d98440ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:49.154433 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cde886-4fbc-42a3-bc11-e59d98440ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:49.221540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-79cde886-4fbc-42a3-bc11-e59d98440ab6 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:49.222748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-79cde886-4fbc-42a3-bc11-e59d98440ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:49.269214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:49.319260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 735/1483] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:13:49.604207 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fcc97-f296-4263-8beb-ed75cf070b24 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:13:49.605791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-401fcc97-f296-4263-8beb-ed75cf070b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:49.629294 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:49.708201 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fcc97-f296-4263-8beb-ed75cf070b24 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:13:49.783584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca639e2-56a5-48a7-be59-dc057971ed32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:13:49.784181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca639e2-56a5-48a7-be59-dc057971ed32 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:49.784456 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca639e2-56a5-48a7-be59-dc057971ed32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:49.784776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca639e2-56a5-48a7-be59-dc057971ed32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:49.786371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-aca639e2-56a5-48a7-be59-dc057971ed32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:49.862197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:49.895765 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca639e2-56a5-48a7-be59-dc057971ed32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:13:49.897233 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca639e2-56a5-48a7-be59-dc057971ed32 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:13:49.914520 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 749/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:13:51.888619 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa17e5e-326e-43d3-9308-4a4c2cd5c299 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:51.889185 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa17e5e-326e-43d3-9308-4a4c2cd5c299 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:51.889185 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa17e5e-326e-43d3-9308-4a4c2cd5c299 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:51.891396 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 736/1485] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:13:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:51.896661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d917b-f959-4446-92e7-3c16bdeec8e3 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:51.897093 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d917b-f959-4446-92e7-3c16bdeec8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:51.897262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d917b-f959-4446-92e7-3c16bdeec8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:51.962930 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-e61d917b-f959-4446-92e7-3c16bdeec8e3 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:51.964055 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e61d917b-f959-4446-92e7-3c16bdeec8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:52.058162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 750/1486] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:13:52.345746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa17e5e-326e-43d3-9308-4a4c2cd5c299 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:13:52.347315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1fa17e5e-326e-43d3-9308-4a4c2cd5c299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:52.450693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa17e5e-326e-43d3-9308-4a4c2cd5c299 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:13:52.522137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:52.523902 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8dda1c-24c6-4690-baf0-d9632758cef3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:13:52.524500 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dda1c-24c6-4690-baf0-d9632758cef3 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:52.524812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dda1c-24c6-4690-baf0-d9632758cef3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:52.525087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dda1c-24c6-4690-baf0-d9632758cef3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:52.526674 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-5a8dda1c-24c6-4690-baf0-d9632758cef3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:52.634197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8dda1c-24c6-4690-baf0-d9632758cef3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:13:52.635540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8dda1c-24c6-4690-baf0-d9632758cef3 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:13:52.653704 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 737/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:13:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:13:54.717952 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d1afd-ebac-4046-a1cb-8b06fe3d080e None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:54.719234 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d1afd-ebac-4046-a1cb-8b06fe3d080e None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:54.719234 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d1afd-ebac-4046-a1cb-8b06fe3d080e None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:54.720566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 751/1488] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:13:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:54.725800 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c83ec-560e-4402-9a5c-64c38567f159 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:54.726236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c83ec-560e-4402-9a5c-64c38567f159 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:54.726358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c83ec-560e-4402-9a5c-64c38567f159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:54.792516 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-223c83ec-560e-4402-9a5c-64c38567f159 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:54.793754 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-223c83ec-560e-4402-9a5c-64c38567f159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:54.846485 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:54.889633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 738/1489] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:13:55.179457 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d1afd-ebac-4046-a1cb-8b06fe3d080e None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:13:55.181043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-233d1afd-ebac-4046-a1cb-8b06fe3d080e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:55.217659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:55.286020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d1afd-ebac-4046-a1cb-8b06fe3d080e None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:13:55.357783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9e304-7c05-46cd-b246-bff7089e7f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:13:55.358377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9e304-7c05-46cd-b246-bff7089e7f0d service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:55.358651 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9e304-7c05-46cd-b246-bff7089e7f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:55.358966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9e304-7c05-46cd-b246-bff7089e7f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:55.360553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-49f9e304-7c05-46cd-b246-bff7089e7f0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:55.438911 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:55.467891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9e304-7c05-46cd-b246-bff7089e7f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:13:55.469662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9e304-7c05-46cd-b246-bff7089e7f0d service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:13:55.487177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 752/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:13:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:13:57.886998 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d18c55-81aa-479a-a101-62924eea06f8 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:57.887548 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d18c55-81aa-479a-a101-62924eea06f8 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:57.887548 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d18c55-81aa-479a-a101-62924eea06f8 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:57.888928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 739/1491] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 12 12:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:13:57.894831 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b1d70-2514-474b-afeb-1a3a880cdc13 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:57.895269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b1d70-2514-474b-afeb-1a3a880cdc13 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:57.895371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b1d70-2514-474b-afeb-1a3a880cdc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:57.961806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-690b1d70-2514-474b-afeb-1a3a880cdc13 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:13:57.963121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-690b1d70-2514-474b-afeb-1a3a880cdc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:58.059445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 753/1492] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 12 12:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:13:58.345869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d18c55-81aa-479a-a101-62924eea06f8 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:13:58.347407 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-29d18c55-81aa-479a-a101-62924eea06f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:58.450812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d18c55-81aa-479a-a101-62924eea06f8 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:13:58.523890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81e794-5217-43db-8632-2eac9ad3809c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:13:58.524483 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81e794-5217-43db-8632-2eac9ad3809c service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:13:58.524798 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81e794-5217-43db-8632-2eac9ad3809c service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:13:58.525070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81e794-5217-43db-8632-2eac9ad3809c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:13:58.526682 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6e81e794-5217-43db-8632-2eac9ad3809c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:13:58.536207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:13:58.634372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81e794-5217-43db-8632-2eac9ad3809c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:13:58.635733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81e794-5217-43db-8632-2eac9ad3809c service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:13:58.655426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 740/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:13:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:14:08.057568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=82 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:14:08.653068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=50 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:15:19.500622 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefdb34b-1afb-408a-b444-36a4fd5957aa None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:19.501232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefdb34b-1afb-408a-b444-36a4fd5957aa None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:19.501232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefdb34b-1afb-408a-b444-36a4fd5957aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:19.576579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-cefdb34b-1afb-408a-b444-36a4fd5957aa None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:15:19.577841 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-cefdb34b-1afb-408a-b444-36a4fd5957aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:19.675841 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 754/1494] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:15:19.684031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f02a5-78d7-4f82-b3f2-9d39e833d235 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:19.684524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f02a5-78d7-4f82-b3f2-9d39e833d235 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:19.684630 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f02a5-78d7-4f82-b3f2-9d39e833d235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:19.760861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-660f02a5-78d7-4f82-b3f2-9d39e833d235 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:15:19.762019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-660f02a5-78d7-4f82-b3f2-9d39e833d235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:19.857495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 741/1495] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:15:19.862244 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefdb34b-1afb-408a-b444-36a4fd5957aa None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:15:19.863479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-cefdb34b-1afb-408a-b444-36a4fd5957aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:19.874751 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefdb34b-1afb-408a-b444-36a4fd5957aa None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:15:19.934798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:15:19.968318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b740a-437b-4839-a664-33d14a30d7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:15:19.968939 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b740a-437b-4839-a664-33d14a30d7fb admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:19.969217 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b740a-437b-4839-a664-33d14a30d7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:19.969484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b740a-437b-4839-a664-33d14a30d7fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:19.970116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b740a-437b-4839-a664-33d14a30d7fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:15:19.970418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b740a-437b-4839-a664-33d14a30d7fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:15:19.971423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: WARNING py.warnings [None req-457b740a-437b-4839-a664-33d14a30d7fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 12 12:15:19.971423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: warnings.warn(msg) Aug 12 12:15:19.971423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: Aug 12 12:15:19.972299 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b740a-437b-4839-a664-33d14a30d7fb admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:15:19.983084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 755/1496] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:15:19.991400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f02a5-78d7-4f82-b3f2-9d39e833d235 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:15:20.000996 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f02a5-78d7-4f82-b3f2-9d39e833d235 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:15:20.062652 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:15:20.096713 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b681c9-9366-4527-b5ec-0b00e1fc98ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:15:20.097316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b681c9-9366-4527-b5ec-0b00e1fc98ff admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:20.097598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b681c9-9366-4527-b5ec-0b00e1fc98ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:20.097868 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b681c9-9366-4527-b5ec-0b00e1fc98ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:20.098490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b681c9-9366-4527-b5ec-0b00e1fc98ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:15:20.100462 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: WARNING py.warnings [None req-29b681c9-9366-4527-b5ec-0b00e1fc98ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 12:15:20.100462 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: warnings.warn(msg) Aug 12 12:15:20.100462 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: Aug 12 12:15:20.101453 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b681c9-9366-4527-b5ec-0b00e1fc98ff admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:15:20.223812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 742/1497] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:15:19 2022] POST /identity/v3/projects => generated 379 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:15:20.228029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b740a-437b-4839-a664-33d14a30d7fb admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:15:20.229581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-457b740a-437b-4839-a664-33d14a30d7fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:20.262863 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b740a-437b-4839-a664-33d14a30d7fb admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:15:20.356258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064b3ae-16ea-423d-8b6c-e88308f7ad58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:15:20.356819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064b3ae-16ea-423d-8b6c-e88308f7ad58 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:20.357103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064b3ae-16ea-423d-8b6c-e88308f7ad58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:20.357377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064b3ae-16ea-423d-8b6c-e88308f7ad58 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:20.358000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064b3ae-16ea-423d-8b6c-e88308f7ad58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:15:20.359022 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: WARNING py.warnings [None req-8064b3ae-16ea-423d-8b6c-e88308f7ad58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 12:15:20.359022 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: warnings.warn(msg) Aug 12 12:15:20.359022 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: Aug 12 12:15:20.360267 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064b3ae-16ea-423d-8b6c-e88308f7ad58 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:15:20.472181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 756/1498] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:15:20 2022] POST /identity/v3/users => generated 290 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:15:20.476304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b681c9-9366-4527-b5ec-0b00e1fc98ff admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:15:20.478453 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-29b681c9-9366-4527-b5ec-0b00e1fc98ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:20.538656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b681c9-9366-4527-b5ec-0b00e1fc98ff admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:15:20.620842 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:15:20.630903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f21e8-9675-4fde-92e5-74e704365cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:15:20.631541 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f21e8-9675-4fde-92e5-74e704365cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:20.631818 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f21e8-9675-4fde-92e5-74e704365cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:20.632084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f21e8-9675-4fde-92e5-74e704365cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:20.632742 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f21e8-9675-4fde-92e5-74e704365cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:15:20.633878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: WARNING py.warnings [None req-7c0f21e8-9675-4fde-92e5-74e704365cf0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 12 12:15:20.633878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: warnings.warn(msg) Aug 12 12:15:20.633878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: Aug 12 12:15:20.634740 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f21e8-9675-4fde-92e5-74e704365cf0 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:15:20.646735 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 743/1499] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:15:20.650964 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064b3ae-16ea-423d-8b6c-e88308f7ad58 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:15:20.652481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-8064b3ae-16ea-423d-8b6c-e88308f7ad58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:20.663813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064b3ae-16ea-423d-8b6c-e88308f7ad58 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:15:20.689692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:15:20.758782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ecd71-fccd-4e20-a29a-d2edfa35fb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:15:20.759379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ecd71-fccd-4e20-a29a-d2edfa35fb8e admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:20.759657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ecd71-fccd-4e20-a29a-d2edfa35fb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:20.759931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ecd71-fccd-4e20-a29a-d2edfa35fb8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:20.760576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ecd71-fccd-4e20-a29a-d2edfa35fb8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:15:20.761631 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: WARNING py.warnings [None req-005ecd71-fccd-4e20-a29a-d2edfa35fb8e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 12 12:15:20.761631 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: warnings.warn(msg) Aug 12 12:15:20.761631 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: Aug 12 12:15:20.762516 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ecd71-fccd-4e20-a29a-d2edfa35fb8e admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:15:20.775161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 757/1500] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:15:20.779311 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f21e8-9675-4fde-92e5-74e704365cf0 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:15:20.780903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-7c0f21e8-9675-4fde-92e5-74e704365cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:20.816417 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f21e8-9675-4fde-92e5-74e704365cf0 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:15:20.912558 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99db22e-a34c-4b76-a8c2-37ddcc587009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:15:20.913243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99db22e-a34c-4b76-a8c2-37ddcc587009 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:20.913524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99db22e-a34c-4b76-a8c2-37ddcc587009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:20.913797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99db22e-a34c-4b76-a8c2-37ddcc587009 admin admin] PATH_INFO: `/v3/projects/8d2a42575ff44f8dad23300e63ce2d7b/users/083a10f8ac3940aabde7a6fed8cc91ba/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:20.936402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99db22e-a34c-4b76-a8c2-37ddcc587009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d2a42575ff44f8dad23300e63ce2d7b, user_id=083a10f8ac3940aabde7a6fed8cc91ba, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:15:20.937470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: WARNING py.warnings [None req-c99db22e-a34c-4b76-a8c2-37ddcc587009 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 12:15:20.937470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: warnings.warn(msg) Aug 12 12:15:20.937470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: Aug 12 12:15:20.939376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99db22e-a34c-4b76-a8c2-37ddcc587009 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:15:21.019864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 744/1501] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:15:20 2022] PUT /identity/v3/projects/8d2a42575ff44f8dad23300e63ce2d7b/users/083a10f8ac3940aabde7a6fed8cc91ba/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:15:21.032439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988275e-d5e2-4cab-a1b6-b377739e6b86 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:21.035957 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988275e-d5e2-4cab-a1b6-b377739e6b86 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:21.035957 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988275e-d5e2-4cab-a1b6-b377739e6b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:21.089265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-1988275e-d5e2-4cab-a1b6-b377739e6b86 None None] MFA Rules not processed for user `083a10f8ac3940aabde7a6fed8cc91ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:15:21.090462 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-1988275e-d5e2-4cab-a1b6-b377739e6b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:21.166974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 758/1502] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:15:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:15:21.180125 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0764dc-14a5-46a6-909f-6954280b8586 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:21.180513 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0764dc-14a5-46a6-909f-6954280b8586 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:21.180822 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0764dc-14a5-46a6-909f-6954280b8586 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:21.182196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 745/1503] 10.222.0.27 () {54 vars in 964 bytes} [Fri Aug 12 12:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:15:21.186746 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa60f23-2fd4-44a3-81a6-2d60fd58f4cb None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:21.187035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa60f23-2fd4-44a3-81a6-2d60fd58f4cb None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:21.187312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa60f23-2fd4-44a3-81a6-2d60fd58f4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:21.218242 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:15:21.261820 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-faa60f23-2fd4-44a3-81a6-2d60fd58f4cb None None] MFA Rules not processed for user `935ab3a263ae46fcb929029a147a1010`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:15:21.263144 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-faa60f23-2fd4-44a3-81a6-2d60fd58f4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:21.338597 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 759/1504] 10.222.0.27 () {58 vars in 1061 bytes} [Fri Aug 12 12:15:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:15:21.346483 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0764dc-14a5-46a6-909f-6954280b8586 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:15:21.348008 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6c0764dc-14a5-46a6-909f-6954280b8586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:21.397169 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:15:21.452127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0764dc-14a5-46a6-909f-6954280b8586 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:15:21.525631 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db05b18-6d02-4254-b9e7-a7dc70d60358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:15:21.526253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db05b18-6d02-4254-b9e7-a7dc70d60358 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:21.526573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db05b18-6d02-4254-b9e7-a7dc70d60358 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:21.526883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db05b18-6d02-4254-b9e7-a7dc70d60358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:21.528587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4db05b18-6d02-4254-b9e7-a7dc70d60358 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:21.627848 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db05b18-6d02-4254-b9e7-a7dc70d60358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:15:21.629301 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db05b18-6d02-4254-b9e7-a7dc70d60358 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:15:21.632797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:15:21.647687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 746/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:15:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:15:21.692859 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec000fb-8853-4fec-94e2-7c3955484dd7 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:21.693283 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec000fb-8853-4fec-94e2-7c3955484dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:21.693426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec000fb-8853-4fec-94e2-7c3955484dd7 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:21.694640 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 760/1506] 10.222.0.27 () {52 vars in 830 bytes} [Fri Aug 12 12:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:15:21.698511 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db05b18-6d02-4254-b9e7-a7dc70d60358 service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:15:21.699760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4db05b18-6d02-4254-b9e7-a7dc70d60358 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:21.797961 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db05b18-6d02-4254-b9e7-a7dc70d60358 service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:15:21.891520 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0c93b-cea3-497a-9ec3-f232fa5d1199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:15:21.892137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0c93b-cea3-497a-9ec3-f232fa5d1199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:21.892419 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0c93b-cea3-497a-9ec3-f232fa5d1199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:21.892776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0c93b-cea3-497a-9ec3-f232fa5d1199 admin admin] PATH_INFO: `/v3/users/083a10f8ac3940aabde7a6fed8cc91ba` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:21.905306 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0c93b-cea3-497a-9ec3-f232fa5d1199 admin admin] RBAC: Authorizing `identity:delete_user(user_id=083a10f8ac3940aabde7a6fed8cc91ba)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:15:21.906327 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: WARNING py.warnings [None req-6fb0c93b-cea3-497a-9ec3-f232fa5d1199 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 12:15:21.906327 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: warnings.warn(msg) Aug 12 12:15:21.906327 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: Aug 12 12:15:21.907529 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0c93b-cea3-497a-9ec3-f232fa5d1199 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:15:21.917111 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:15:22.114373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6fb0c93b-cea3-497a-9ec3-f232fa5d1199 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '083a10f8ac3940aabde7a6fed8cc91ba'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:15:22.114373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6fb0c93b-cea3-497a-9ec3-f232fa5d1199 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '083a10f8ac3940aabde7a6fed8cc91ba'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:15:22.177174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6fb0c93b-cea3-497a-9ec3-f232fa5d1199 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '083a10f8ac3940aabde7a6fed8cc91ba'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:15:22.200806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6fb0c93b-cea3-497a-9ec3-f232fa5d1199 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '083a10f8ac3940aabde7a6fed8cc91ba'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:15:22.215043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6fb0c93b-cea3-497a-9ec3-f232fa5d1199 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '083a10f8ac3940aabde7a6fed8cc91ba'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:15:22.220541 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 747/1507] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:15:21 2022] DELETE /identity/v3/users/083a10f8ac3940aabde7a6fed8cc91ba => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:15:22.224888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962af02f-1c3c-4480-815f-efbcb2fbaff6 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:22.225176 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962af02f-1c3c-4480-815f-efbcb2fbaff6 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:22.225451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962af02f-1c3c-4480-815f-efbcb2fbaff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:22.281227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-962af02f-1c3c-4480-815f-efbcb2fbaff6 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:15:22.282417 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-962af02f-1c3c-4480-815f-efbcb2fbaff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:22.358200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:15:22.370797 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 761/1508] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:15:22.380159 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0c93b-cea3-497a-9ec3-f232fa5d1199 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:15:22.381418 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6fb0c93b-cea3-497a-9ec3-f232fa5d1199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:22.481606 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0c93b-cea3-497a-9ec3-f232fa5d1199 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:15:22.537068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:15:22.556859 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4549ca6-3c1d-4ddc-93d1-8f620d4966e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:15:22.557471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4549ca6-3c1d-4ddc-93d1-8f620d4966e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:22.557757 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4549ca6-3c1d-4ddc-93d1-8f620d4966e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:22.558034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4549ca6-3c1d-4ddc-93d1-8f620d4966e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:22.559633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f4549ca6-3c1d-4ddc-93d1-8f620d4966e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:22.668124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4549ca6-3c1d-4ddc-93d1-8f620d4966e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:15:22.669581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4549ca6-3c1d-4ddc-93d1-8f620d4966e6 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:15:22.688203 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 748/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:15:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:15:23.157177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962af02f-1c3c-4480-815f-efbcb2fbaff6 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:15:23.158443 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-962af02f-1c3c-4480-815f-efbcb2fbaff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:15:23.170809 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962af02f-1c3c-4480-815f-efbcb2fbaff6 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:15:23.268464 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd4eb5-2142-4a31-8027-036feae94abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:15:23.269083 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd4eb5-2142-4a31-8027-036feae94abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:15:23.269365 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd4eb5-2142-4a31-8027-036feae94abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:15:23.269640 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd4eb5-2142-4a31-8027-036feae94abf admin admin] PATH_INFO: `/v3/projects/8d2a42575ff44f8dad23300e63ce2d7b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:15:23.280044 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd4eb5-2142-4a31-8027-036feae94abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d2a42575ff44f8dad23300e63ce2d7b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:15:23.281098 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: WARNING py.warnings [None req-defd4eb5-2142-4a31-8027-036feae94abf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 12:15:23.281098 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: warnings.warn(msg) Aug 12 12:15:23.281098 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: Aug 12 12:15:23.282290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd4eb5-2142-4a31-8027-036feae94abf admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:15:23.451280 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-defd4eb5-2142-4a31-8027-036feae94abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d2a42575ff44f8dad23300e63ce2d7b', 'request_id': 'req-defd4eb5-2142-4a31-8027-036feae94abf'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:15:23.457244 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-defd4eb5-2142-4a31-8027-036feae94abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d2a42575ff44f8dad23300e63ce2d7b', 'request_id': 'req-defd4eb5-2142-4a31-8027-036feae94abf'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:15:23.470945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-defd4eb5-2142-4a31-8027-036feae94abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d2a42575ff44f8dad23300e63ce2d7b', 'request_id': 'req-defd4eb5-2142-4a31-8027-036feae94abf'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:15:23.579808 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-defd4eb5-2142-4a31-8027-036feae94abf admin admin] The token cache is being invalidate because project 8d2a42575ff44f8dad23300e63ce2d7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:15:23.580018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-defd4eb5-2142-4a31-8027-036feae94abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:15:23.580543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-defd4eb5-2142-4a31-8027-036feae94abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:15:23.581911 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 762/1510] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:15:23 2022] DELETE /identity/v3/projects/8d2a42575ff44f8dad23300e63ce2d7b => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:15:32.686023 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=62 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:15:33.577766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:16:51.097305 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce01702-ba6f-4e20-a0c2-bd8d4feb8524 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:16:51.097867 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce01702-ba6f-4e20-a0c2-bd8d4feb8524 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:16:51.097867 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce01702-ba6f-4e20-a0c2-bd8d4feb8524 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:16:51.099128 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 12:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:16:51.104222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a768089-afe7-40e2-ab91-98cb82456066 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:16:51.104694 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a768089-afe7-40e2-ab91-98cb82456066 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:16:51.104808 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a768089-afe7-40e2-ab91-98cb82456066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:16:51.183959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-5a768089-afe7-40e2-ab91-98cb82456066 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:16:51.185191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-5a768089-afe7-40e2-ab91-98cb82456066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:16:51.285699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 12:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:16:51.442438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce01702-ba6f-4e20-a0c2-bd8d4feb8524 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:16:51.444722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-7ce01702-ba6f-4e20-a0c2-bd8d4feb8524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:16:51.565510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce01702-ba6f-4e20-a0c2-bd8d4feb8524 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:16:51.571060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1ad36-7aaa-479d-b943-b289b4d94bab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:16:51.571695 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1ad36-7aaa-479d-b943-b289b4d94bab service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:16:51.571979 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1ad36-7aaa-479d-b943-b289b4d94bab service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:16:51.572254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1ad36-7aaa-479d-b943-b289b4d94bab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:16:51.580665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1ad36-7aaa-479d-b943-b289b4d94bab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:16:51.582251 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1ad36-7aaa-479d-b943-b289b4d94bab service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:16:51.592313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 750/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:16:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:16:52.432791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6519b57-f2e6-4f16-8d53-6d1b919544ff None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:16:52.432791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6519b57-f2e6-4f16-8d53-6d1b919544ff None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:16:52.432791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6519b57-f2e6-4f16-8d53-6d1b919544ff None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:16:52.432791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 12:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:16:52.433605 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a5e67-4e6c-41c7-a17b-6facf368e0f3 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:16:52.433605 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a5e67-4e6c-41c7-a17b-6facf368e0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:16:52.433605 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a5e67-4e6c-41c7-a17b-6facf368e0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:16:52.453239 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-f36a5e67-4e6c-41c7-a17b-6facf368e0f3 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:16:52.454451 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f36a5e67-4e6c-41c7-a17b-6facf368e0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:16:52.462356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 12:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:16:52.614884 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6519b57-f2e6-4f16-8d53-6d1b919544ff None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:16:52.629950 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6519b57-f2e6-4f16-8d53-6d1b919544ff None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:16:52.635379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4595aa-0856-4140-a617-363595e4a8c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:16:52.635970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4595aa-0856-4140-a617-363595e4a8c8 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:16:52.636251 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4595aa-0856-4140-a617-363595e4a8c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:16:52.636524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4595aa-0856-4140-a617-363595e4a8c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:16:52.644487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4595aa-0856-4140-a617-363595e4a8c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:16:52.646100 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4595aa-0856-4140-a617-363595e4a8c8 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:16:52.655327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 765/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:16:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:16:52.973610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b6658-01dc-44f2-b2a3-d4a419246550 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:16:52.973810 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b6658-01dc-44f2-b2a3-d4a419246550 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:16:52.974073 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b6658-01dc-44f2-b2a3-d4a419246550 None None] PATH_INFO: `/v3` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:16:52.975373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 752/1517] 10.222.0.46 () {54 vars in 925 bytes} [Fri Aug 12 12:16:52 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:16:52.982051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4d195-8bcc-444d-9acc-f3a336ac70e4 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:16:52.982339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4d195-8bcc-444d-9acc-f3a336ac70e4 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:16:52.982619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4d195-8bcc-444d-9acc-f3a336ac70e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:16:53.041384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-14c4d195-8bcc-444d-9acc-f3a336ac70e4 None None] MFA Rules not processed for user `5ee8ba21bc184feeaafacab3e69e87bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:16:53.042617 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-14c4d195-8bcc-444d-9acc-f3a336ac70e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:16:53.059724 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:16:53.114637 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 766/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Fri Aug 12 12:16:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:17:02.448530 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=56 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:17:03.111921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=23 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:17:03.860906 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb4443-80b5-48fa-b488-c6815b2bf136 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:03.861382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb4443-80b5-48fa-b488-c6815b2bf136 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:03.861382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb4443-80b5-48fa-b488-c6815b2bf136 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:03.862580 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a777ee-1038-4d2a-a6f3-19d9f7a1a0f5 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:03.862855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 753/1519] 10.222.0.27 () {54 vars in 961 bytes} [Fri Aug 12 12:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:17:03.863047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a777ee-1038-4d2a-a6f3-19d9f7a1a0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:03.863162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a777ee-1038-4d2a-a6f3-19d9f7a1a0f5 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:03.864434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 767/1520] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 12:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 12 12:17:03.867723 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064528b-da4c-4141-a1df-f72d266b88b7 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:03.868009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064528b-da4c-4141-a1df-f72d266b88b7 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:03.868283 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064528b-da4c-4141-a1df-f72d266b88b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:03.868997 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127305fe-e295-43d4-a281-e22960cd2fa6 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:03.869362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127305fe-e295-43d4-a281-e22960cd2fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:03.869722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127305fe-e295-43d4-a281-e22960cd2fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:03.888261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-8064528b-da4c-4141-a1df-f72d266b88b7 None None] MFA Rules not processed for user `5ee8ba21bc184feeaafacab3e69e87bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:17:03.888479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-127305fe-e295-43d4-a281-e22960cd2fa6 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:17:03.889574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8064528b-da4c-4141-a1df-f72d266b88b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:17:03.889689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-127305fe-e295-43d4-a281-e22960cd2fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:17:03.897390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 768/1521] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 12:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:17:03.898298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 754/1522] 10.222.0.27 () {58 vars in 1058 bytes} [Fri Aug 12 12:17:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:17:03.906370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127305fe-e295-43d4-a281-e22960cd2fa6 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:17:03.917514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127305fe-e295-43d4-a281-e22960cd2fa6 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:17:03.922745 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8a726-e6f8-4669-9248-f6344be4d468 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ee8ba21bc184feeaafacab3e69e87bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:17:03.923330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8a726-e6f8-4669-9248-f6344be4d468 service placement] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:03.923608 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8a726-e6f8-4669-9248-f6344be4d468 service placement] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:03.923880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8a726-e6f8-4669-9248-f6344be4d468 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:03.932427 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8a726-e6f8-4669-9248-f6344be4d468 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:17:03.934091 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8a726-e6f8-4669-9248-f6344be4d468 service placement] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:17:03.943774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 769/1523] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:17:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:17:04.053915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064528b-da4c-4141-a1df-f72d266b88b7 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:17:04.066329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064528b-da4c-4141-a1df-f72d266b88b7 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:17:04.071805 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19a648-03d4-4c6b-9117-966b27711edf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:17:04.073003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19a648-03d4-4c6b-9117-966b27711edf service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:04.073284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19a648-03d4-4c6b-9117-966b27711edf service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:04.073557 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19a648-03d4-4c6b-9117-966b27711edf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:04.081846 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19a648-03d4-4c6b-9117-966b27711edf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:17:04.083407 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19a648-03d4-4c6b-9117-966b27711edf service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:17:04.093315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 755/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:17:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:17:04.927857 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defa61b-8b96-430c-a0b5-6215387617c4 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:04.928336 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defa61b-8b96-430c-a0b5-6215387617c4 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:04.928428 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defa61b-8b96-430c-a0b5-6215387617c4 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:04.929798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 12 12:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:17:04.935434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d43d8-6df7-4547-a40a-d18456ac79d2 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:04.935869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d43d8-6df7-4547-a40a-d18456ac79d2 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:04.935964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d43d8-6df7-4547-a40a-d18456ac79d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:04.956746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-829d43d8-6df7-4547-a40a-d18456ac79d2 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:17:04.958712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-829d43d8-6df7-4547-a40a-d18456ac79d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:17:04.969086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 12 12:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:17:05.123635 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defa61b-8b96-430c-a0b5-6215387617c4 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:17:05.137532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defa61b-8b96-430c-a0b5-6215387617c4 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:17:05.142803 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652619d-08be-4ad0-a24f-c30f53de3592 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:17:05.143388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652619d-08be-4ad0-a24f-c30f53de3592 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:05.143666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652619d-08be-4ad0-a24f-c30f53de3592 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:05.143932 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652619d-08be-4ad0-a24f-c30f53de3592 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:05.151858 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652619d-08be-4ad0-a24f-c30f53de3592 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:17:05.153477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652619d-08be-4ad0-a24f-c30f53de3592 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:17:05.162590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 771/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:17:08.921897 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67acf477-0cf0-4a04-bb9a-a2435b64920b None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:08.922409 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67acf477-0cf0-4a04-bb9a-a2435b64920b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:08.922503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67acf477-0cf0-4a04-bb9a-a2435b64920b None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:08.923826 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 12:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:17:08.928821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcc464-a844-446c-a3e2-e3422e21dd6b None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:08.929284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcc464-a844-446c-a3e2-e3422e21dd6b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:08.929382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcc464-a844-446c-a3e2-e3422e21dd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:08.948449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-5bfcc464-a844-446c-a3e2-e3422e21dd6b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:17:08.949659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-5bfcc464-a844-446c-a3e2-e3422e21dd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:17:08.957291 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 12:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:17:09.253043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67acf477-0cf0-4a04-bb9a-a2435b64920b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:17:09.254728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-67acf477-0cf0-4a04-bb9a-a2435b64920b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:17:09.331468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67acf477-0cf0-4a04-bb9a-a2435b64920b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:17:09.334983 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d56995-c13d-4c1d-944a-798921f96e2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:17:09.335571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d56995-c13d-4c1d-944a-798921f96e2a service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:09.335850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d56995-c13d-4c1d-944a-798921f96e2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:09.336118 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d56995-c13d-4c1d-944a-798921f96e2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:09.345367 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d56995-c13d-4c1d-944a-798921f96e2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:17:09.346796 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d56995-c13d-4c1d-944a-798921f96e2a service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:17:09.356547 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 758/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:17:11.246861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342eae6-459c-490b-98b9-ff9a7133a86b None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:11.247420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342eae6-459c-490b-98b9-ff9a7133a86b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:11.247420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342eae6-459c-490b-98b9-ff9a7133a86b None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:11.248786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 12:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:17:11.253780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e01a4-1e69-402f-ae35-f6be8b7782be None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:11.254188 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e01a4-1e69-402f-ae35-f6be8b7782be None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:11.254342 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e01a4-1e69-402f-ae35-f6be8b7782be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:11.273289 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-126e01a4-1e69-402f-ae35-f6be8b7782be None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:17:11.274466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-126e01a4-1e69-402f-ae35-f6be8b7782be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:17:11.282378 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 12:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:17:11.569113 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342eae6-459c-490b-98b9-ff9a7133a86b None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:17:11.580925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342eae6-459c-490b-98b9-ff9a7133a86b None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:17:11.584713 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7444e0-3ffc-45b6-b200-3c04e42712f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:17:11.585381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7444e0-3ffc-45b6-b200-3c04e42712f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:11.585706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7444e0-3ffc-45b6-b200-3c04e42712f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:11.586021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7444e0-3ffc-45b6-b200-3c04e42712f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:11.595520 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7444e0-3ffc-45b6-b200-3c04e42712f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:17:11.597065 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7444e0-3ffc-45b6-b200-3c04e42712f6 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:17:11.607202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 774/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:17:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:17:14.916760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4697e-8c05-4ee6-a402-6bfe439f043b None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:14.917313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4697e-8c05-4ee6-a402-6bfe439f043b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:14.917313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4697e-8c05-4ee6-a402-6bfe439f043b None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:14.918782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 12:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:17:14.924192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6da92-07f0-4c47-a5c1-cc2058b48fcf None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:14.924619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6da92-07f0-4c47-a5c1-cc2058b48fcf None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:14.924821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6da92-07f0-4c47-a5c1-cc2058b48fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:14.944132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-8ba6da92-07f0-4c47-a5c1-cc2058b48fcf None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:17:14.945367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-8ba6da92-07f0-4c47-a5c1-cc2058b48fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:17:14.953838 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 12:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:17:15.504920 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4697e-8c05-4ee6-a402-6bfe439f043b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:17:15.506678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-08f4697e-8c05-4ee6-a402-6bfe439f043b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:17:15.584801 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4697e-8c05-4ee6-a402-6bfe439f043b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:17:15.588402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bda916-0f7c-4659-b96e-44b05977394f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:17:15.589021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bda916-0f7c-4659-b96e-44b05977394f service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:15.589299 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bda916-0f7c-4659-b96e-44b05977394f service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:15.589576 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bda916-0f7c-4659-b96e-44b05977394f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:15.598389 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bda916-0f7c-4659-b96e-44b05977394f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:17:15.599803 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bda916-0f7c-4659-b96e-44b05977394f service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:17:15.609589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 761/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:17:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:17:17.603376 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fca66-737e-4b79-8e7d-c4604efdc0b1 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:17.603918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fca66-737e-4b79-8e7d-c4604efdc0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:17.603918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6fca66-737e-4b79-8e7d-c4604efdc0b1 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:17.605260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 12 12:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:17:17.611017 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216e040-8e2b-46f0-8f16-ec04c59a12b2 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:17.611533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216e040-8e2b-46f0-8f16-ec04c59a12b2 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:17.611632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216e040-8e2b-46f0-8f16-ec04c59a12b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:17.631999 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-f216e040-8e2b-46f0-8f16-ec04c59a12b2 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:17:17.633276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f216e040-8e2b-46f0-8f16-ec04c59a12b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:17:17.642207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 12 12:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:17:18.222883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fca66-737e-4b79-8e7d-c4604efdc0b1 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:17:18.224597 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7e6fca66-737e-4b79-8e7d-c4604efdc0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:17:18.238196 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6fca66-737e-4b79-8e7d-c4604efdc0b1 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:17:18.241785 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3d4a5-fa75-4820-841a-6bcb8c40017f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:17:18.242381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3d4a5-fa75-4820-841a-6bcb8c40017f service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:17:18.242660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3d4a5-fa75-4820-841a-6bcb8c40017f service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:17:18.243533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3d4a5-fa75-4820-841a-6bcb8c40017f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:17:18.252836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3d4a5-fa75-4820-841a-6bcb8c40017f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:17:18.254277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3d4a5-fa75-4820-841a-6bcb8c40017f service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:17:18.263742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 777/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:17:27.626964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=88 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:17:28.259522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=33 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:42.938351 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485324b-e3c2-42b1-9e04-51ce44b6ab1b None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:42.939023 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485324b-e3c2-42b1-9e04-51ce44b6ab1b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:42.939023 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485324b-e3c2-42b1-9e04-51ce44b6ab1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:42.970060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-9485324b-e3c2-42b1-9e04-51ce44b6ab1b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:42.971347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9485324b-e3c2-42b1-9e04-51ce44b6ab1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:42.986635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 763/1540] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:42.996170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d473005-9b4f-4cce-a9c3-340fc218efdd None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:42.996796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d473005-9b4f-4cce-a9c3-340fc218efdd None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:42.996796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d473005-9b4f-4cce-a9c3-340fc218efdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.002162 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed9233-e52e-460e-b23d-9913b367b25d None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.002444 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed9233-e52e-460e-b23d-9913b367b25d None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.002721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed9233-e52e-460e-b23d-9913b367b25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.037618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-4d473005-9b4f-4cce-a9c3-340fc218efdd None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:43.038854 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4d473005-9b4f-4cce-a9c3-340fc218efdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:43.053003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-56ed9233-e52e-460e-b23d-9913b367b25d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:43.054201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-56ed9233-e52e-460e-b23d-9913b367b25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:43.060784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 778/1541] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:43.065612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d473005-9b4f-4cce-a9c3-340fc218efdd None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.073754 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 764/1542] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:43.078107 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d473005-9b4f-4cce-a9c3-340fc218efdd None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.086241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f113e-87ae-4ee0-815f-df57cf9198c5 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.086536 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f113e-87ae-4ee0-815f-df57cf9198c5 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.086819 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f113e-87ae-4ee0-815f-df57cf9198c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.096833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d744ec9-5784-432a-bd16-4dc3f91331b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.097460 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d744ec9-5784-432a-bd16-4dc3f91331b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.097743 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d744ec9-5784-432a-bd16-4dc3f91331b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.098112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d744ec9-5784-432a-bd16-4dc3f91331b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.098921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d744ec9-5784-432a-bd16-4dc3f91331b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:18:43.099425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d744ec9-5784-432a-bd16-4dc3f91331b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.102686 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d744ec9-5784-432a-bd16-4dc3f91331b6 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.125842 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-990f113e-87ae-4ee0-815f-df57cf9198c5 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:43.127072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-990f113e-87ae-4ee0-815f-df57cf9198c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:43.137954 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 779/1543] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:18:43.145303 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d744ec9-5784-432a-bd16-4dc3f91331b6 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.148050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 765/1544] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:43.162349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d744ec9-5784-432a-bd16-4dc3f91331b6 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.164717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f113e-87ae-4ee0-815f-df57cf9198c5 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.175952 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ebdfc-cc17-4dd6-ab2a-f8cf73600aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.177153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ebdfc-cc17-4dd6-ab2a-f8cf73600aba admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.177450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ebdfc-cc17-4dd6-ab2a-f8cf73600aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.178041 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ebdfc-cc17-4dd6-ab2a-f8cf73600aba admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.178684 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ebdfc-cc17-4dd6-ab2a-f8cf73600aba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.179715 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: WARNING py.warnings [None req-a28ebdfc-cc17-4dd6-ab2a-f8cf73600aba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 12:18:43.179715 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: warnings.warn(msg) Aug 12 12:18:43.179715 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: Aug 12 12:18:43.181071 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ebdfc-cc17-4dd6-ab2a-f8cf73600aba admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.183167 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f113e-87ae-4ee0-815f-df57cf9198c5 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.200244 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7bc4d-b9ac-48d7-91c6-23bda61de068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.200896 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7bc4d-b9ac-48d7-91c6-23bda61de068 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.201184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7bc4d-b9ac-48d7-91c6-23bda61de068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.202125 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7bc4d-b9ac-48d7-91c6-23bda61de068 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.202806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7bc4d-b9ac-48d7-91c6-23bda61de068 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:18:43.203126 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7bc4d-b9ac-48d7-91c6-23bda61de068 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.204705 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: WARNING py.warnings [None req-d2e7bc4d-b9ac-48d7-91c6-23bda61de068 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 12 12:18:43.204705 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: warnings.warn(msg) Aug 12 12:18:43.204705 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: Aug 12 12:18:43.205796 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7bc4d-b9ac-48d7-91c6-23bda61de068 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.228361 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 766/1545] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:18:43.241748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7bc4d-b9ac-48d7-91c6-23bda61de068 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.258907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7bc4d-b9ac-48d7-91c6-23bda61de068 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.266443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcca1dc-5cbe-4828-a869-9043b5761de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.267116 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcca1dc-5cbe-4828-a869-9043b5761de9 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.267405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcca1dc-5cbe-4828-a869-9043b5761de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.267678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcca1dc-5cbe-4828-a869-9043b5761de9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.268321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcca1dc-5cbe-4828-a869-9043b5761de9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.270309 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcca1dc-5cbe-4828-a869-9043b5761de9 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.331533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 780/1546] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/projects => generated 371 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:43.333539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a3c41-3658-458f-b6ab-5c77f7a2ef18 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.333829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a3c41-3658-458f-b6ab-5c77f7a2ef18 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.334103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a3c41-3658-458f-b6ab-5c77f7a2ef18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.352594 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-a37a3c41-3658-458f-b6ab-5c77f7a2ef18 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:43.356565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a37a3c41-3658-458f-b6ab-5c77f7a2ef18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:43.411134 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 781/1547] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:43.415017 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 767/1548] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:43.415171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77feb4a-2b8f-40c7-878d-573a08672dba None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.415171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77feb4a-2b8f-40c7-878d-573a08672dba None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.415171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77feb4a-2b8f-40c7-878d-573a08672dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.415386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc1feb-8bbe-40e0-ac8e-16b79ef82f6e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.415386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc1feb-8bbe-40e0-ac8e-16b79ef82f6e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.415386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc1feb-8bbe-40e0-ac8e-16b79ef82f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.434075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-7ebc1feb-8bbe-40e0-ac8e-16b79ef82f6e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:43.434309 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f77feb4a-2b8f-40c7-878d-573a08672dba None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:43.435274 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-7ebc1feb-8bbe-40e0-ac8e-16b79ef82f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:43.435400 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f77feb4a-2b8f-40c7-878d-573a08672dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:43.496395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 782/1549] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:43.496725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 768/1550] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:43.498093 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb0252-1c2b-40f4-8e52-bf02ba28807e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.498231 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d65c4c-83d9-4690-9b61-8c26a55a1b4b None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.498426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb0252-1c2b-40f4-8e52-bf02ba28807e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.498583 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d65c4c-83d9-4690-9b61-8c26a55a1b4b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.498667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb0252-1c2b-40f4-8e52-bf02ba28807e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.498991 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d65c4c-83d9-4690-9b61-8c26a55a1b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.521861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-a6d65c4c-83d9-4690-9b61-8c26a55a1b4b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:43.522970 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-9beb0252-1c2b-40f4-8e52-bf02ba28807e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:43.523162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a6d65c4c-83d9-4690-9b61-8c26a55a1b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:43.524142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9beb0252-1c2b-40f4-8e52-bf02ba28807e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:43.531830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 783/1551] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:43.531968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 769/1552] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:43.532628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb0252-1c2b-40f4-8e52-bf02ba28807e None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.538155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbf325-185b-402a-a172-88215bfa3fd1 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.539546 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbf325-185b-402a-a172-88215bfa3fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.540074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbf325-185b-402a-a172-88215bfa3fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.543406 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb0252-1c2b-40f4-8e52-bf02ba28807e None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.548085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a439c-a3e0-4656-ad29-87966aa9bcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.548705 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a439c-a3e0-4656-ad29-87966aa9bcd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.548994 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a439c-a3e0-4656-ad29-87966aa9bcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.549298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a439c-a3e0-4656-ad29-87966aa9bcd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.549917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a439c-a3e0-4656-ad29-87966aa9bcd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.550931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: WARNING py.warnings [None req-365a439c-a3e0-4656-ad29-87966aa9bcd0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 12:18:43.550931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: warnings.warn(msg) Aug 12 12:18:43.550931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: Aug 12 12:18:43.552164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a439c-a3e0-4656-ad29-87966aa9bcd0 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.559667 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-49dbf325-185b-402a-a172-88215bfa3fd1 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:43.561805 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-49dbf325-185b-402a-a172-88215bfa3fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:43.569450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 784/1553] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:43.570204 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbf325-185b-402a-a172-88215bfa3fd1 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.579612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbf325-185b-402a-a172-88215bfa3fd1 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.586790 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab87b9-d3ba-4f54-9008-55bca835447e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.587385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab87b9-d3ba-4f54-9008-55bca835447e admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.587663 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab87b9-d3ba-4f54-9008-55bca835447e admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.587931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab87b9-d3ba-4f54-9008-55bca835447e admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.588620 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab87b9-d3ba-4f54-9008-55bca835447e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.590642 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab87b9-d3ba-4f54-9008-55bca835447e admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.665053 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 770/1554] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/users => generated 286 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:43.666378 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1465f-3f3b-4e94-86cf-33d2eec6ce57 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.666924 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1465f-3f3b-4e94-86cf-33d2eec6ce57 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.667199 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1465f-3f3b-4e94-86cf-33d2eec6ce57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.687238 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-60f1465f-3f3b-4e94-86cf-33d2eec6ce57 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:43.688450 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-60f1465f-3f3b-4e94-86cf-33d2eec6ce57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:43.691232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 785/1555] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:43.693402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b24ca1-6e64-4fe8-b465-590972ed6795 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.693503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b24ca1-6e64-4fe8-b465-590972ed6795 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.693747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b24ca1-6e64-4fe8-b465-590972ed6795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.698339 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 771/1556] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:43.699538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080820f-6b2c-43d1-93e6-8987dbad2f28 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.699827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080820f-6b2c-43d1-93e6-8987dbad2f28 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.700106 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080820f-6b2c-43d1-93e6-8987dbad2f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.717171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-80b24ca1-6e64-4fe8-b465-590972ed6795 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:43.718371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-80b24ca1-6e64-4fe8-b465-590972ed6795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:43.719719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-6080820f-6b2c-43d1-93e6-8987dbad2f28 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:43.720988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6080820f-6b2c-43d1-93e6-8987dbad2f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:43.726605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 786/1557] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:43.727779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c6007-57dc-4309-8383-fc09e9d05b52 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.728086 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c6007-57dc-4309-8383-fc09e9d05b52 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.728373 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c6007-57dc-4309-8383-fc09e9d05b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.732099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 772/1558] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:43.733326 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fe527-b3ec-49cb-91e8-af32fe1a1890 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.733625 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fe527-b3ec-49cb-91e8-af32fe1a1890 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.733892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fe527-b3ec-49cb-91e8-af32fe1a1890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.751344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-ef1c6007-57dc-4309-8383-fc09e9d05b52 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:43.752492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ef1c6007-57dc-4309-8383-fc09e9d05b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:43.753815 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-366fe527-b3ec-49cb-91e8-af32fe1a1890 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:43.755888 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-366fe527-b3ec-49cb-91e8-af32fe1a1890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:43.761164 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 787/1559] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:43.762386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc84714-93de-4245-9023-f9c58440ba91 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.762703 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc84714-93de-4245-9023-f9c58440ba91 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.762975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc84714-93de-4245-9023-f9c58440ba91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.767634 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 773/1560] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:43.767634 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fe527-b3ec-49cb-91e8-af32fe1a1890 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.775435 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fe527-b3ec-49cb-91e8-af32fe1a1890 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.778597 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:43.780003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80038156-a2f1-491a-9cde-2440e89d4e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.780676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80038156-a2f1-491a-9cde-2440e89d4e07 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.780875 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80038156-a2f1-491a-9cde-2440e89d4e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.781147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80038156-a2f1-491a-9cde-2440e89d4e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.781780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80038156-a2f1-491a-9cde-2440e89d4e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.783421 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80038156-a2f1-491a-9cde-2440e89d4e07 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.784553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-bdc84714-93de-4245-9023-f9c58440ba91 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:43.785827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-bdc84714-93de-4245-9023-f9c58440ba91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:43.793998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 788/1561] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:43.797525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 774/1562] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:43.797673 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc84714-93de-4245-9023-f9c58440ba91 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.797781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80038156-a2f1-491a-9cde-2440e89d4e07 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.808169 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80038156-a2f1-491a-9cde-2440e89d4e07 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.812441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc84714-93de-4245-9023-f9c58440ba91 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.812906 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a43e28-01f9-4316-995b-988f3e1a2b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.813880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a43e28-01f9-4316-995b-988f3e1a2b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.813880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a43e28-01f9-4316-995b-988f3e1a2b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.814071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a43e28-01f9-4316-995b-988f3e1a2b50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.814699 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a43e28-01f9-4316-995b-988f3e1a2b50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:18:43.814999 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a43e28-01f9-4316-995b-988f3e1a2b50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.816573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a43e28-01f9-4316-995b-988f3e1a2b50 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.817831 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847a95e-91f3-4bca-9be7-a7f6a2d594ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.818423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847a95e-91f3-4bca-9be7-a7f6a2d594ae admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.818701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847a95e-91f3-4bca-9be7-a7f6a2d594ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.818979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847a95e-91f3-4bca-9be7-a7f6a2d594ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.819638 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847a95e-91f3-4bca-9be7-a7f6a2d594ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.821256 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847a95e-91f3-4bca-9be7-a7f6a2d594ae admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.829747 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 775/1563] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:18:43.830542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a43e28-01f9-4316-995b-988f3e1a2b50 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.831361 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 789/1564] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:43.834850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847a95e-91f3-4bca-9be7-a7f6a2d594ae admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.842190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a43e28-01f9-4316-995b-988f3e1a2b50 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.846502 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847a95e-91f3-4bca-9be7-a7f6a2d594ae admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.846715 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac67c5-4cb7-4cf8-ba9e-68040128afc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.847300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac67c5-4cb7-4cf8-ba9e-68040128afc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.847577 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac67c5-4cb7-4cf8-ba9e-68040128afc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.847883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac67c5-4cb7-4cf8-ba9e-68040128afc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.848549 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac67c5-4cb7-4cf8-ba9e-68040128afc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:18:43.848881 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac67c5-4cb7-4cf8-ba9e-68040128afc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.850454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac67c5-4cb7-4cf8-ba9e-68040128afc3 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.851265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0febf33-b6d6-4226-9c3a-a954dcd9a541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.851847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0febf33-b6d6-4226-9c3a-a954dcd9a541 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.852127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0febf33-b6d6-4226-9c3a-a954dcd9a541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.852396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0febf33-b6d6-4226-9c3a-a954dcd9a541 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.853066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0febf33-b6d6-4226-9c3a-a954dcd9a541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:18:43.853364 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0febf33-b6d6-4226-9c3a-a954dcd9a541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.854926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0febf33-b6d6-4226-9c3a-a954dcd9a541 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.862115 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 776/1565] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:18:43.862902 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac67c5-4cb7-4cf8-ba9e-68040128afc3 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.869983 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 790/1566] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:18:43.870832 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0febf33-b6d6-4226-9c3a-a954dcd9a541 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.873289 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac67c5-4cb7-4cf8-ba9e-68040128afc3 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.878564 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cfaf6-0fc7-4e79-a39e-7b5cd81053ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.879147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cfaf6-0fc7-4e79-a39e-7b5cd81053ef admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.880009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cfaf6-0fc7-4e79-a39e-7b5cd81053ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.880288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cfaf6-0fc7-4e79-a39e-7b5cd81053ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.880956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114cfaf6-0fc7-4e79-a39e-7b5cd81053ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:18:43.881256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114cfaf6-0fc7-4e79-a39e-7b5cd81053ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.881637 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0febf33-b6d6-4226-9c3a-a954dcd9a541 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.882825 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114cfaf6-0fc7-4e79-a39e-7b5cd81053ef admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.886405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9285451-6179-4395-9426-a7a762c27343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.886986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9285451-6179-4395-9426-a7a762c27343 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.887268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9285451-6179-4395-9426-a7a762c27343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.887543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9285451-6179-4395-9426-a7a762c27343 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.888182 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9285451-6179-4395-9426-a7a762c27343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:18:43.888489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9285451-6179-4395-9426-a7a762c27343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.890128 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9285451-6179-4395-9426-a7a762c27343 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.894762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 777/1567] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:18:43.897543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cfaf6-0fc7-4e79-a39e-7b5cd81053ef admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.902467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 791/1568] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:18:43.903557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9285451-6179-4395-9426-a7a762c27343 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.913033 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cfaf6-0fc7-4e79-a39e-7b5cd81053ef admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.913329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9285451-6179-4395-9426-a7a762c27343 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.917767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb10d14-16e9-4285-b6d2-03c9adca4bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.918100 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d76a70-6397-485e-8e87-9e059fa85ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.918351 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb10d14-16e9-4285-b6d2-03c9adca4bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.918629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb10d14-16e9-4285-b6d2-03c9adca4bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.918733 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d76a70-6397-485e-8e87-9e059fa85ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.918900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb10d14-16e9-4285-b6d2-03c9adca4bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.919022 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d76a70-6397-485e-8e87-9e059fa85ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.919287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d76a70-6397-485e-8e87-9e059fa85ca2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.919529 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb10d14-16e9-4285-b6d2-03c9adca4bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.919921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d76a70-6397-485e-8e87-9e059fa85ca2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:18:43.920221 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d76a70-6397-485e-8e87-9e059fa85ca2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.921164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb10d14-16e9-4285-b6d2-03c9adca4bb2 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.922030 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d76a70-6397-485e-8e87-9e059fa85ca2 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.927438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:43.932122 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 778/1569] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:43.932847 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb10d14-16e9-4285-b6d2-03c9adca4bb2 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.939875 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 792/1570] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:18:43.939875 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d76a70-6397-485e-8e87-9e059fa85ca2 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.948542 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d76a70-6397-485e-8e87-9e059fa85ca2 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.951006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb10d14-16e9-4285-b6d2-03c9adca4bb2 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.953134 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e67216-5788-4afc-aa3d-659f213b5a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.953751 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e67216-5788-4afc-aa3d-659f213b5a09 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.954027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e67216-5788-4afc-aa3d-659f213b5a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.954298 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e67216-5788-4afc-aa3d-659f213b5a09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.954909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e67216-5788-4afc-aa3d-659f213b5a09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.956175 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b63a1a-099c-49bd-ae81-811793f65e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.956801 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b63a1a-099c-49bd-ae81-811793f65e8c admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.956915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e67216-5788-4afc-aa3d-659f213b5a09 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.957180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b63a1a-099c-49bd-ae81-811793f65e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.957992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b63a1a-099c-49bd-ae81-811793f65e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.958956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b63a1a-099c-49bd-ae81-811793f65e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.961230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b63a1a-099c-49bd-ae81-811793f65e8c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:43.971566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 779/1571] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:43.972337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b63a1a-099c-49bd-ae81-811793f65e8c admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:43.981703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b63a1a-099c-49bd-ae81-811793f65e8c admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:43.986161 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af767e3d-44a5-49e3-9197-b218d2d2b3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:43.986730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af767e3d-44a5-49e3-9197-b218d2d2b3cb admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:43.987008 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af767e3d-44a5-49e3-9197-b218d2d2b3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:43.987279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af767e3d-44a5-49e3-9197-b218d2d2b3cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:43.987934 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af767e3d-44a5-49e3-9197-b218d2d2b3cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:43.990319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af767e3d-44a5-49e3-9197-b218d2d2b3cb admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:44.048780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 793/1572] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:44.049590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e67216-5788-4afc-aa3d-659f213b5a09 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:44.077448 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 780/1573] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:18:43 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:44.078250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af767e3d-44a5-49e3-9197-b218d2d2b3cb admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:44.120293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e67216-5788-4afc-aa3d-659f213b5a09 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:44.140773 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af767e3d-44a5-49e3-9197-b218d2d2b3cb admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:44.145573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d35ee7-33a8-44f0-8cb9-3594fb543f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:44.146179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d35ee7-33a8-44f0-8cb9-3594fb543f14 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:44.146457 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d35ee7-33a8-44f0-8cb9-3594fb543f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:44.146726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d35ee7-33a8-44f0-8cb9-3594fb543f14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:44.147350 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d35ee7-33a8-44f0-8cb9-3594fb543f14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:44.149283 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d35ee7-33a8-44f0-8cb9-3594fb543f14 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:44.177615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540def6-f3c1-4133-aa17-a4b16ab7fc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:44.178436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540def6-f3c1-4133-aa17-a4b16ab7fc35 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:44.178840 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540def6-f3c1-4133-aa17-a4b16ab7fc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:44.179234 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540def6-f3c1-4133-aa17-a4b16ab7fc35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:44.180163 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540def6-f3c1-4133-aa17-a4b16ab7fc35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:44.182116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540def6-f3c1-4133-aa17-a4b16ab7fc35 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:44.266235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 781/1574] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:18:44 2022] POST /identity/v3/projects => generated 389 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:44.267048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d35ee7-33a8-44f0-8cb9-3594fb543f14 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:44.295793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 794/1575] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:18:44 2022] POST /identity/v3/projects => generated 383 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:44.296618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540def6-f3c1-4133-aa17-a4b16ab7fc35 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:44.318765 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:44.330616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d35ee7-33a8-44f0-8cb9-3594fb543f14 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:44.359938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540def6-f3c1-4133-aa17-a4b16ab7fc35 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:44.365237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab253c47-d525-453b-bbe9-3343a05a9ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:44.368264 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab253c47-d525-453b-bbe9-3343a05a9ebf admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:44.368264 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab253c47-d525-453b-bbe9-3343a05a9ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:44.368264 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab253c47-d525-453b-bbe9-3343a05a9ebf admin admin] PATH_INFO: `/v3/projects/29877b59841f428f9d094a309fdce828/users/253488caedb44ea4b3b7c1c8f5dc8731/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:44.382079 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab253c47-d525-453b-bbe9-3343a05a9ebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=29877b59841f428f9d094a309fdce828, user_id=253488caedb44ea4b3b7c1c8f5dc8731, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:44.383126 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: WARNING py.warnings [None req-ab253c47-d525-453b-bbe9-3343a05a9ebf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 12:18:44.383126 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: warnings.warn(msg) Aug 12 12:18:44.383126 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: Aug 12 12:18:44.385115 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab253c47-d525-453b-bbe9-3343a05a9ebf admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:44.391311 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91beba-2452-4843-86cd-b588ae2e5544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:44.391965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91beba-2452-4843-86cd-b588ae2e5544 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:44.392230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91beba-2452-4843-86cd-b588ae2e5544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:44.392505 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91beba-2452-4843-86cd-b588ae2e5544 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:44.393185 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91beba-2452-4843-86cd-b588ae2e5544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:44.395793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91beba-2452-4843-86cd-b588ae2e5544 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:44.400355 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:44.450537 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 795/1576] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:18:44 2022] PUT /identity/v3/projects/29877b59841f428f9d094a309fdce828/users/253488caedb44ea4b3b7c1c8f5dc8731/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:44.451354 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab253c47-d525-453b-bbe9-3343a05a9ebf admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:44.476376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 782/1577] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:18:44 2022] POST /identity/v3/projects => generated 379 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:44.477496 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91beba-2452-4843-86cd-b588ae2e5544 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:44.517575 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab253c47-d525-453b-bbe9-3343a05a9ebf admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:44.538612 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91beba-2452-4843-86cd-b588ae2e5544 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:44.543517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3287d-7a97-4d64-b076-02ef011d4c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:44.544171 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3287d-7a97-4d64-b076-02ef011d4c87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:44.544446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3287d-7a97-4d64-b076-02ef011d4c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:44.544749 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3287d-7a97-4d64-b076-02ef011d4c87 admin admin] PATH_INFO: `/v3/projects/9ad47eec693748db9c3c0303fc8cc747/users/0e30ba9436f940188d7fed18c8ca9dae/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:44.572543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3287d-7a97-4d64-b076-02ef011d4c87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ad47eec693748db9c3c0303fc8cc747, user_id=0e30ba9436f940188d7fed18c8ca9dae, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:44.574584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7621c-0d08-4255-899e-ff2be30f1bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:44.575183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3287d-7a97-4d64-b076-02ef011d4c87 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:44.575305 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7621c-0d08-4255-899e-ff2be30f1bee admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:44.575460 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7621c-0d08-4255-899e-ff2be30f1bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:44.575731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7621c-0d08-4255-899e-ff2be30f1bee admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:44.576349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d7621c-0d08-4255-899e-ff2be30f1bee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:44.578254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d7621c-0d08-4255-899e-ff2be30f1bee admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:44.673353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 783/1578] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:18:44 2022] PUT /identity/v3/projects/9ad47eec693748db9c3c0303fc8cc747/users/0e30ba9436f940188d7fed18c8ca9dae/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:44.674277 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3287d-7a97-4d64-b076-02ef011d4c87 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:44.698594 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 796/1579] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:18:44 2022] POST /identity/v3/projects => generated 379 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:44.699334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7621c-0d08-4255-899e-ff2be30f1bee admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:44.740164 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:44.740365 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3287d-7a97-4d64-b076-02ef011d4c87 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:44.762328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7621c-0d08-4255-899e-ff2be30f1bee admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:44.767508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03d532-d3a2-40af-8526-9fd30f6a1c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:44.768114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03d532-d3a2-40af-8526-9fd30f6a1c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:44.768398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03d532-d3a2-40af-8526-9fd30f6a1c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:44.768699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03d532-d3a2-40af-8526-9fd30f6a1c3f admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:44.769321 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03d532-d3a2-40af-8526-9fd30f6a1c3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:44.771370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03d532-d3a2-40af-8526-9fd30f6a1c3f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:44.797122 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41434f4-e653-48ab-9c57-b8fc84c3b439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:44.797731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41434f4-e653-48ab-9c57-b8fc84c3b439 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:44.798010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41434f4-e653-48ab-9c57-b8fc84c3b439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:44.798282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41434f4-e653-48ab-9c57-b8fc84c3b439 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:44.798948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41434f4-e653-48ab-9c57-b8fc84c3b439 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:44.800889 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41434f4-e653-48ab-9c57-b8fc84c3b439 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:44.833359 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 797/1580] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:18:44 2022] POST /identity/v3/users => generated 297 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:44.834115 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03d532-d3a2-40af-8526-9fd30f6a1c3f admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:44.843550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03d532-d3a2-40af-8526-9fd30f6a1c3f admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:44.848671 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658de7ab-e49d-4d21-a62f-bdf08b96690b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:44.849343 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658de7ab-e49d-4d21-a62f-bdf08b96690b admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:44.849624 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658de7ab-e49d-4d21-a62f-bdf08b96690b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:44.849895 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658de7ab-e49d-4d21-a62f-bdf08b96690b admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:44.850498 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658de7ab-e49d-4d21-a62f-bdf08b96690b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:44.852409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658de7ab-e49d-4d21-a62f-bdf08b96690b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:44.907521 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 784/1581] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:18:44 2022] POST /identity/v3/users => generated 284 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:44.907521 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41434f4-e653-48ab-9c57-b8fc84c3b439 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:44.911963 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:44.915917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41434f4-e653-48ab-9c57-b8fc84c3b439 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:44.920759 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e11a8a-4bbb-422f-8a6a-b88712694402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:44.921330 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e11a8a-4bbb-422f-8a6a-b88712694402 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:44.921609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e11a8a-4bbb-422f-8a6a-b88712694402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:44.921880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e11a8a-4bbb-422f-8a6a-b88712694402 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:44.922541 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e11a8a-4bbb-422f-8a6a-b88712694402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:44.924411 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e11a8a-4bbb-422f-8a6a-b88712694402 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:44.933923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 798/1582] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:18:44 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:44.935268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7234e0-f72d-4143-91d7-7d3dab07d83b None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:44.935547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7234e0-f72d-4143-91d7-7d3dab07d83b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:44.936582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7234e0-f72d-4143-91d7-7d3dab07d83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:44.964904 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-5e7234e0-f72d-4143-91d7-7d3dab07d83b None None] MFA Rules not processed for user `253488caedb44ea4b3b7c1c8f5dc8731`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:44.966809 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-5e7234e0-f72d-4143-91d7-7d3dab07d83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:45.010752 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 785/1583] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:18:44 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:45.011600 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e11a8a-4bbb-422f-8a6a-b88712694402 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.030958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 799/1584] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:45.032373 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38746452-8ff7-4a11-b097-44e4fd7ca0c1 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.032766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38746452-8ff7-4a11-b097-44e4fd7ca0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.033116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38746452-8ff7-4a11-b097-44e4fd7ca0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.044129 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e11a8a-4bbb-422f-8a6a-b88712694402 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.049192 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79869fea-06fb-45c3-84e0-9ca2e9e76dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.050636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79869fea-06fb-45c3-84e0-9ca2e9e76dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.051831 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79869fea-06fb-45c3-84e0-9ca2e9e76dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.052092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79869fea-06fb-45c3-84e0-9ca2e9e76dc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.052929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79869fea-06fb-45c3-84e0-9ca2e9e76dc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.055050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79869fea-06fb-45c3-84e0-9ca2e9e76dc0 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.071019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-38746452-8ff7-4a11-b097-44e4fd7ca0c1 None None] MFA Rules not processed for user `0e30ba9436f940188d7fed18c8ca9dae`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:45.072468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-38746452-8ff7-4a11-b097-44e4fd7ca0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:45.136419 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 786/1585] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:18:45 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:45.137421 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79869fea-06fb-45c3-84e0-9ca2e9e76dc0 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.142360 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 800/1586] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:45.142360 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38746452-8ff7-4a11-b097-44e4fd7ca0c1 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.150005 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79869fea-06fb-45c3-84e0-9ca2e9e76dc0 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.151394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38746452-8ff7-4a11-b097-44e4fd7ca0c1 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.155000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a68a0e-ab91-4ea6-832a-d454b3923845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.155911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a68a0e-ab91-4ea6-832a-d454b3923845 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.158319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb98ba6-bfae-44e3-ab91-f7073a8d2949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.159331 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a68a0e-ab91-4ea6-832a-d454b3923845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.159331 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a68a0e-ab91-4ea6-832a-d454b3923845 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.159494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb98ba6-bfae-44e3-ab91-f7073a8d2949 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.159494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb98ba6-bfae-44e3-ab91-f7073a8d2949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.159494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb98ba6-bfae-44e3-ab91-f7073a8d2949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.159727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a68a0e-ab91-4ea6-832a-d454b3923845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.159803 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb98ba6-bfae-44e3-ab91-f7073a8d2949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.159875 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a68a0e-ab91-4ea6-832a-d454b3923845 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.159953 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb98ba6-bfae-44e3-ab91-f7073a8d2949 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.174584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 801/1587] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:45.174584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb98ba6-bfae-44e3-ab91-f7073a8d2949 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.182888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb98ba6-bfae-44e3-ab91-f7073a8d2949 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.187603 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd275ae-d803-4b3c-a0b8-55ad324b7b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.188182 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd275ae-d803-4b3c-a0b8-55ad324b7b2d admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.188458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd275ae-d803-4b3c-a0b8-55ad324b7b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.188756 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd275ae-d803-4b3c-a0b8-55ad324b7b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.189389 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd275ae-d803-4b3c-a0b8-55ad324b7b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.190967 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd275ae-d803-4b3c-a0b8-55ad324b7b2d admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.200528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 802/1588] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:45.201401 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd275ae-d803-4b3c-a0b8-55ad324b7b2d admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.207698 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:45.211490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd275ae-d803-4b3c-a0b8-55ad324b7b2d admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.216106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ae5a1-251b-4aaf-9526-27724cc56bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.216762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ae5a1-251b-4aaf-9526-27724cc56bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.217108 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ae5a1-251b-4aaf-9526-27724cc56bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.217388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ae5a1-251b-4aaf-9526-27724cc56bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.218015 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ae5a1-251b-4aaf-9526-27724cc56bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.219575 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ae5a1-251b-4aaf-9526-27724cc56bd0 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.230590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 803/1589] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:45.231351 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ae5a1-251b-4aaf-9526-27724cc56bd0 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.240931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ae5a1-251b-4aaf-9526-27724cc56bd0 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.245124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 787/1590] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:18:45 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:45.245124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f6700-5362-4722-b5c6-ba763e7545cb None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.245124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f6700-5362-4722-b5c6-ba763e7545cb None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.245124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f6700-5362-4722-b5c6-ba763e7545cb None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.248926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35068f90-4e53-4467-9f9c-60c2f593f5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.249317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 788/1591] 10.222.0.27 () {54 vars in 962 bytes} [Fri Aug 12 12:18:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:18:45.249904 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35068f90-4e53-4467-9f9c-60c2f593f5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.250068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f6700-5362-4722-b5c6-ba763e7545cb None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.250190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35068f90-4e53-4467-9f9c-60c2f593f5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.250465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35068f90-4e53-4467-9f9c-60c2f593f5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.251168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35068f90-4e53-4467-9f9c-60c2f593f5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.252785 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35068f90-4e53-4467-9f9c-60c2f593f5c0 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.263053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 804/1592] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:45.264760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f6700-5362-4722-b5c6-ba763e7545cb None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.264871 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820f7f6-a06f-4eec-bc5e-61aab0af6f5b None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.264871 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820f7f6-a06f-4eec-bc5e-61aab0af6f5b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.264871 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820f7f6-a06f-4eec-bc5e-61aab0af6f5b None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.271383 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 805/1593] 10.222.0.27 () {54 vars in 962 bytes} [Fri Aug 12 12:18:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:18:45.271383 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820f7f6-a06f-4eec-bc5e-61aab0af6f5b None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.271539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac4f4c-a9b8-4615-a8b4-c46ce998942e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.271539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac4f4c-a9b8-4615-a8b4-c46ce998942e admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.271539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac4f4c-a9b8-4615-a8b4-c46ce998942e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.271539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac4f4c-a9b8-4615-a8b4-c46ce998942e admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.271890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac4f4c-a9b8-4615-a8b4-c46ce998942e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.272756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac4f4c-a9b8-4615-a8b4-c46ce998942e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.282057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820f7f6-a06f-4eec-bc5e-61aab0af6f5b None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.286480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 789/1594] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:45.286480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac4f4c-a9b8-4615-a8b4-c46ce998942e admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.287470 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25da002-11cc-4412-8ff0-73a2735e4778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.288064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25da002-11cc-4412-8ff0-73a2735e4778 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.288357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25da002-11cc-4412-8ff0-73a2735e4778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.288621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25da002-11cc-4412-8ff0-73a2735e4778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.289322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25da002-11cc-4412-8ff0-73a2735e4778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.290886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25da002-11cc-4412-8ff0-73a2735e4778 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.294312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac4f4c-a9b8-4615-a8b4-c46ce998942e admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.298852 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a59a8-ef64-4b74-bb23-d6eef65dbdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.299428 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a59a8-ef64-4b74-bb23-d6eef65dbdbc admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.299708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a59a8-ef64-4b74-bb23-d6eef65dbdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.299978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a59a8-ef64-4b74-bb23-d6eef65dbdbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.300608 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a59a8-ef64-4b74-bb23-d6eef65dbdbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.302303 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a59a8-ef64-4b74-bb23-d6eef65dbdbc admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.305750 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 806/1595] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:45.305750 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25da002-11cc-4412-8ff0-73a2735e4778 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.314874 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25da002-11cc-4412-8ff0-73a2735e4778 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.316455 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 790/1596] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:45.317518 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a59a8-ef64-4b74-bb23-d6eef65dbdbc admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.320455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa74fbb-fa05-452d-b6b1-e277e18186be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.321183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa74fbb-fa05-452d-b6b1-e277e18186be admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.321580 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa74fbb-fa05-452d-b6b1-e277e18186be admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.325990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa74fbb-fa05-452d-b6b1-e277e18186be admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.325990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa74fbb-fa05-452d-b6b1-e277e18186be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.325990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa74fbb-fa05-452d-b6b1-e277e18186be admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.328029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a59a8-ef64-4b74-bb23-d6eef65dbdbc admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.333261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03b0d6-3715-4aad-bc35-b28a670545d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.333855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03b0d6-3715-4aad-bc35-b28a670545d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.334137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03b0d6-3715-4aad-bc35-b28a670545d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.334407 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03b0d6-3715-4aad-bc35-b28a670545d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.335042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03b0d6-3715-4aad-bc35-b28a670545d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.335345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 807/1597] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:45.336591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a999a-3fec-4c7c-9e45-99cb4cd90d27 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.336750 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03b0d6-3715-4aad-bc35-b28a670545d2 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.336993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a999a-3fec-4c7c-9e45-99cb4cd90d27 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.337269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a999a-3fec-4c7c-9e45-99cb4cd90d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.350706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 791/1598] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:45.350706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3744bd-632a-44a5-88f8-0df7abb7d07b None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.350706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3744bd-632a-44a5-88f8-0df7abb7d07b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.350706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3744bd-632a-44a5-88f8-0df7abb7d07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.376146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-213a999a-3fec-4c7c-9e45-99cb4cd90d27 None None] MFA Rules not processed for user `687d0375abd44d2481f988c6df90fad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:45.377330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-213a999a-3fec-4c7c-9e45-99cb4cd90d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:45.378912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-1d3744bd-632a-44a5-88f8-0df7abb7d07b None None] MFA Rules not processed for user `687d0375abd44d2481f988c6df90fad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:45.383349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1d3744bd-632a-44a5-88f8-0df7abb7d07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:45.446065 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 808/1599] 10.222.0.27 () {58 vars in 1059 bytes} [Fri Aug 12 12:18:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:45.446921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a999a-3fec-4c7c-9e45-99cb4cd90d27 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.449525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 792/1600] 10.222.0.27 () {58 vars in 1059 bytes} [Fri Aug 12 12:18:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:45.450577 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3744bd-632a-44a5-88f8-0df7abb7d07b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.458541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a999a-3fec-4c7c-9e45-99cb4cd90d27 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.463089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3744bd-632a-44a5-88f8-0df7abb7d07b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.463235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe409e82-b861-46e5-9ad3-e1053e4ab27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.463860 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe409e82-b861-46e5-9ad3-e1053e4ab27f admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.464138 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe409e82-b861-46e5-9ad3-e1053e4ab27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.464409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe409e82-b861-46e5-9ad3-e1053e4ab27f admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.465080 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe409e82-b861-46e5-9ad3-e1053e4ab27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.466667 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe409e82-b861-46e5-9ad3-e1053e4ab27f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.467809 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb84b8-044f-4636-bd6a-57c4df6f49b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.468407 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb84b8-044f-4636-bd6a-57c4df6f49b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.468710 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb84b8-044f-4636-bd6a-57c4df6f49b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.468990 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb84b8-044f-4636-bd6a-57c4df6f49b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.469629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb84b8-044f-4636-bd6a-57c4df6f49b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.471207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb84b8-044f-4636-bd6a-57c4df6f49b2 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.478300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 809/1601] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:45.479095 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe409e82-b861-46e5-9ad3-e1053e4ab27f admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.485505 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 793/1602] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:45.486314 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb84b8-044f-4636-bd6a-57c4df6f49b2 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.488935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe409e82-b861-46e5-9ad3-e1053e4ab27f admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.493693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba361655-c332-414e-8ef2-9c277e3df1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.500847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba361655-c332-414e-8ef2-9c277e3df1bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.500847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba361655-c332-414e-8ef2-9c277e3df1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.500847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba361655-c332-414e-8ef2-9c277e3df1bd admin admin] PATH_INFO: `/v3/projects/2e8def85acdd4e308bb51b4063c2ab4e/users/099b87a6c6b8409ab188c0b71ded7f55/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.501114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb84b8-044f-4636-bd6a-57c4df6f49b2 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.501114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b75025-59c4-4962-8f82-aa25a4efc22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.501556 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b75025-59c4-4962-8f82-aa25a4efc22b admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.501840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b75025-59c4-4962-8f82-aa25a4efc22b admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.502131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b75025-59c4-4962-8f82-aa25a4efc22b admin admin] PATH_INFO: `/v3/projects/d526623066fb41aa8c2d8e09e7b6858c/users/f62af74954884d0693a4877a869f02fb/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.509216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba361655-c332-414e-8ef2-9c277e3df1bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e8def85acdd4e308bb51b4063c2ab4e, user_id=099b87a6c6b8409ab188c0b71ded7f55, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.511851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba361655-c332-414e-8ef2-9c277e3df1bd admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.517005 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b75025-59c4-4962-8f82-aa25a4efc22b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d526623066fb41aa8c2d8e09e7b6858c, user_id=f62af74954884d0693a4877a869f02fb, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.519606 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b75025-59c4-4962-8f82-aa25a4efc22b admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.573594 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 810/1603] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:18:45 2022] PUT /identity/v3/projects/2e8def85acdd4e308bb51b4063c2ab4e/users/099b87a6c6b8409ab188c0b71ded7f55/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:45.574393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba361655-c332-414e-8ef2-9c277e3df1bd admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.596268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 794/1604] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:18:45 2022] PUT /identity/v3/projects/d526623066fb41aa8c2d8e09e7b6858c/users/f62af74954884d0693a4877a869f02fb/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:45.597116 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b75025-59c4-4962-8f82-aa25a4efc22b admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.629864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:45.641797 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba361655-c332-414e-8ef2-9c277e3df1bd admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.650509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:45.661303 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b75025-59c4-4962-8f82-aa25a4efc22b admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.667356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaec744-d208-436c-8f82-00c4aa912e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.667356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaec744-d208-436c-8f82-00c4aa912e8d admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.667356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaec744-d208-436c-8f82-00c4aa912e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.667667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaec744-d208-436c-8f82-00c4aa912e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.668033 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaec744-d208-436c-8f82-00c4aa912e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.669679 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaec744-d208-436c-8f82-00c4aa912e8d admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.679270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 795/1605] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:45.680063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaec744-d208-436c-8f82-00c4aa912e8d admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.698309 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc610e-fb0a-49ed-ad0a-cce557d7ae5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.699085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc610e-fb0a-49ed-ad0a-cce557d7ae5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.699369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc610e-fb0a-49ed-ad0a-cce557d7ae5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.699671 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc610e-fb0a-49ed-ad0a-cce557d7ae5e admin admin] PATH_INFO: `/v3/projects/f84d5684e1904e569a5551b9e21ce0ff/users/383cbf83186747a5b81a12ed5288c39a/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.714501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc610e-fb0a-49ed-ad0a-cce557d7ae5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84d5684e1904e569a5551b9e21ce0ff, user_id=383cbf83186747a5b81a12ed5288c39a, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.717500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc610e-fb0a-49ed-ad0a-cce557d7ae5e admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.749709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaec744-d208-436c-8f82-00c4aa912e8d admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.755833 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3cd0a-ed35-45e5-81f8-e29e6ec2c5d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.756435 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3cd0a-ed35-45e5-81f8-e29e6ec2c5d7 service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.756749 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3cd0a-ed35-45e5-81f8-e29e6ec2c5d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.757011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3cd0a-ed35-45e5-81f8-e29e6ec2c5d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.807866 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 811/1606] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:18:45 2022] PUT /identity/v3/projects/f84d5684e1904e569a5551b9e21ce0ff/users/383cbf83186747a5b81a12ed5288c39a/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:45.809003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc610e-fb0a-49ed-ad0a-cce557d7ae5e admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.814673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3cd0a-ed35-45e5-81f8-e29e6ec2c5d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.816262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3cd0a-ed35-45e5-81f8-e29e6ec2c5d7 service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.826205 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 796/1607] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:18:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:18:45.827045 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3cd0a-ed35-45e5-81f8-e29e6ec2c5d7 service nova] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.837938 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3cd0a-ed35-45e5-81f8-e29e6ec2c5d7 service nova] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.889158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc610e-fb0a-49ed-ad0a-cce557d7ae5e admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.895824 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cf0d0-7e8c-4009-9849-3014a0ae99a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.896595 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cf0d0-7e8c-4009-9849-3014a0ae99a5 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.896909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cf0d0-7e8c-4009-9849-3014a0ae99a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.897184 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cf0d0-7e8c-4009-9849-3014a0ae99a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.898853 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1194da-19a6-4a11-96e9-3d7b3a818801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:45.899504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1194da-19a6-4a11-96e9-3d7b3a818801 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.899779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1194da-19a6-4a11-96e9-3d7b3a818801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.900264 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1194da-19a6-4a11-96e9-3d7b3a818801 admin admin] PATH_INFO: `/v3/projects/3d79773c49d643fc8eb04d69aafb0b01/users/0156c689e109489d9265dd3dd4d5cfde/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:45.904278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:45.916328 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1194da-19a6-4a11-96e9-3d7b3a818801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d79773c49d643fc8eb04d69aafb0b01, user_id=0156c689e109489d9265dd3dd4d5cfde, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.920022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1194da-19a6-4a11-96e9-3d7b3a818801 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.954802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83cf0d0-7e8c-4009-9849-3014a0ae99a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:45.956390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83cf0d0-7e8c-4009-9849-3014a0ae99a5 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:45.966991 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 812/1608] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:18:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:18:45.973730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cf0d0-7e8c-4009-9849-3014a0ae99a5 service nova] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:45.986440 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cf0d0-7e8c-4009-9849-3014a0ae99a5 service nova] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:45.991797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 797/1609] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:18:45 2022] PUT /identity/v3/projects/3d79773c49d643fc8eb04d69aafb0b01/users/0156c689e109489d9265dd3dd4d5cfde/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:45.997081 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0afc5-475d-4e66-a9e9-4a0e843c997e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:45.997370 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0afc5-475d-4e66-a9e9-4a0e843c997e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:45.997644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0afc5-475d-4e66-a9e9-4a0e843c997e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.018105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-56c0afc5-475d-4e66-a9e9-4a0e843c997e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:46.019860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-56c0afc5-475d-4e66-a9e9-4a0e843c997e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:46.034848 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:46.072560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e56952-bed2-4fa0-b0d7-d00b07cb9286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:46.073962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e56952-bed2-4fa0-b0d7-d00b07cb9286 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.074244 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e56952-bed2-4fa0-b0d7-d00b07cb9286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.074576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e56952-bed2-4fa0-b0d7-d00b07cb9286 admin admin] PATH_INFO: `/v3/projects/fbc30c5ff3ea4fcbb7b176a430f18cf3/users/aab3f62ff9bc461fa47a34d851a86642/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.087919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 798/1610] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:46.089588 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52ce12-1511-4c2c-9130-1fea5ab4255d None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.090051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52ce12-1511-4c2c-9130-1fea5ab4255d None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.090489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52ce12-1511-4c2c-9130-1fea5ab4255d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.092521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e56952-bed2-4fa0-b0d7-d00b07cb9286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc30c5ff3ea4fcbb7b176a430f18cf3, user_id=aab3f62ff9bc461fa47a34d851a86642, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:46.095699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e56952-bed2-4fa0-b0d7-d00b07cb9286 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:46.119878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-8a52ce12-1511-4c2c-9130-1fea5ab4255d None None] MFA Rules not processed for user `f62af74954884d0693a4877a869f02fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:46.121156 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8a52ce12-1511-4c2c-9130-1fea5ab4255d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:46.168939 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 813/1611] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:18:45 2022] PUT /identity/v3/projects/fbc30c5ff3ea4fcbb7b176a430f18cf3/users/aab3f62ff9bc461fa47a34d851a86642/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:46.169958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e56952-bed2-4fa0-b0d7-d00b07cb9286 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:46.190262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 799/1612] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:46.191476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e4c96-a5ee-45a8-8a97-1f40e7956ca2 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.192470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e4c96-a5ee-45a8-8a97-1f40e7956ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.192904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e4c96-a5ee-45a8-8a97-1f40e7956ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.226820 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-2b3e4c96-a5ee-45a8-8a97-1f40e7956ca2 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:46.226820 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2b3e4c96-a5ee-45a8-8a97-1f40e7956ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:46.260052 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e56952-bed2-4fa0-b0d7-d00b07cb9286 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:46.264927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff1a7f-74fd-4a36-bc34-e89086a4cd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:46.265582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff1a7f-74fd-4a36-bc34-e89086a4cd8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.265903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff1a7f-74fd-4a36-bc34-e89086a4cd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.266186 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff1a7f-74fd-4a36-bc34-e89086a4cd8c admin admin] PATH_INFO: `/v3/projects/a765c7ccb93545e0a90bc7ffc23a0d46/users/9bc7ffa8e3cf49fe9dcd8471c25be83a/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.281684 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff1a7f-74fd-4a36-bc34-e89086a4cd8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a765c7ccb93545e0a90bc7ffc23a0d46, user_id=9bc7ffa8e3cf49fe9dcd8471c25be83a, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:46.284328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff1a7f-74fd-4a36-bc34-e89086a4cd8c admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:46.299634 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 800/1613] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:46.301633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b13322-a769-47ce-83c7-6af1289814fd None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.303077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b13322-a769-47ce-83c7-6af1289814fd None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.303077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b13322-a769-47ce-83c7-6af1289814fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.321020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-c1b13322-a769-47ce-83c7-6af1289814fd None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:46.322221 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c1b13322-a769-47ce-83c7-6af1289814fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:46.330037 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 801/1614] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:46.331871 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6dd83-8142-4868-ad3a-f19fc68ead94 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.331871 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6dd83-8142-4868-ad3a-f19fc68ead94 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.332057 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6dd83-8142-4868-ad3a-f19fc68ead94 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.333357 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 802/1615] 10.222.0.27 () {54 vars in 912 bytes} [Fri Aug 12 12:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:18:46.334064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6dd83-8142-4868-ad3a-f19fc68ead94 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:46.349399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:46.356444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 814/1616] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:18:46 2022] PUT /identity/v3/projects/a765c7ccb93545e0a90bc7ffc23a0d46/users/9bc7ffa8e3cf49fe9dcd8471c25be83a/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:46.357909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4041024-5092-4129-a108-6b04987c105e None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.358196 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4041024-5092-4129-a108-6b04987c105e None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.360562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4041024-5092-4129-a108-6b04987c105e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.386237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-b4041024-5092-4129-a108-6b04987c105e None None] MFA Rules not processed for user `aab3f62ff9bc461fa47a34d851a86642`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:46.387447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b4041024-5092-4129-a108-6b04987c105e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:46.397538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6dd83-8142-4868-ad3a-f19fc68ead94 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:46.448409 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d3fc6-0d5b-4369-a845-cbd497fe7e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:46.448975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d3fc6-0d5b-4369-a845-cbd497fe7e07 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.449255 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d3fc6-0d5b-4369-a845-cbd497fe7e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.449732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d3fc6-0d5b-4369-a845-cbd497fe7e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.452998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 815/1617] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:46.453835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4041024-5092-4129-a108-6b04987c105e None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:46.483112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:46.507607 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4041024-5092-4129-a108-6b04987c105e None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:46.511419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c9e14-9e9b-4933-b0f2-fa885341697e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:46.512015 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c9e14-9e9b-4933-b0f2-fa885341697e service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.512294 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c9e14-9e9b-4933-b0f2-fa885341697e service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.512589 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c9e14-9e9b-4933-b0f2-fa885341697e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.518624 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d3fc6-0d5b-4369-a845-cbd497fe7e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:46.520673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d3fc6-0d5b-4369-a845-cbd497fe7e07 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:46.530345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 803/1618] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:18:46.534012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388493d-af3d-4707-a643-5bb7aaa1fde8 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.534381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388493d-af3d-4707-a643-5bb7aaa1fde8 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.534728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388493d-af3d-4707-a643-5bb7aaa1fde8 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.536582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 804/1619] 10.222.0.27 () {54 vars in 912 bytes} [Fri Aug 12 12:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:18:46.537474 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388493d-af3d-4707-a643-5bb7aaa1fde8 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:46.549950 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388493d-af3d-4707-a643-5bb7aaa1fde8 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:46.554319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f807e-43ed-4120-91b9-d64efc9190ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:46.554980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f807e-43ed-4120-91b9-d64efc9190ae service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.555264 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f807e-43ed-4120-91b9-d64efc9190ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.555533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f807e-43ed-4120-91b9-d64efc9190ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.564950 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f807e-43ed-4120-91b9-d64efc9190ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:46.566391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f807e-43ed-4120-91b9-d64efc9190ae service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:46.576308 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 805/1620] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:18:46.577507 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0789dc0-78b7-4878-8fcd-a558a8df9094 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.577788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0789dc0-78b7-4878-8fcd-a558a8df9094 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.578062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0789dc0-78b7-4878-8fcd-a558a8df9094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.578972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c9e14-9e9b-4933-b0f2-fa885341697e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:46.580435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c9e14-9e9b-4933-b0f2-fa885341697e service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:46.591489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 816/1621] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:18:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:18:46.592798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c9e14-9e9b-4933-b0f2-fa885341697e service cinder] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:46.598939 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-b0789dc0-78b7-4878-8fcd-a558a8df9094 None None] MFA Rules not processed for user `687d0375abd44d2481f988c6df90fad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:46.600212 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b0789dc0-78b7-4878-8fcd-a558a8df9094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:46.602172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c9e14-9e9b-4933-b0f2-fa885341697e service cinder] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:46.606093 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2317b3-cb70-494b-b54a-06b6fc31d6af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:46.612328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2317b3-cb70-494b-b54a-06b6fc31d6af service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.612328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2317b3-cb70-494b-b54a-06b6fc31d6af service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.612328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2317b3-cb70-494b-b54a-06b6fc31d6af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.618105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2317b3-cb70-494b-b54a-06b6fc31d6af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:46.619731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2317b3-cb70-494b-b54a-06b6fc31d6af service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:46.634807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 817/1622] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:18:46.634807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e752ab-5dba-4640-9a88-221509824df3 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.634807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e752ab-5dba-4640-9a88-221509824df3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.634807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e752ab-5dba-4640-9a88-221509824df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.663033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-67e752ab-5dba-4640-9a88-221509824df3 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:46.664241 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-67e752ab-5dba-4640-9a88-221509824df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:46.669746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:46.673194 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 818/1623] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:46.684696 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e752ab-5dba-4640-9a88-221509824df3 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:46.690094 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 806/1624] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Aug 12 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:46.693470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949568a-f40a-403c-b47b-73f4c3513ba0 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.694164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949568a-f40a-403c-b47b-73f4c3513ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.694688 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949568a-f40a-403c-b47b-73f4c3513ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.701818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e752ab-5dba-4640-9a88-221509824df3 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:46.707624 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a19f2-270d-4553-b687-019f5d8b912d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:46.708299 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a19f2-270d-4553-b687-019f5d8b912d admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.708689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a19f2-270d-4553-b687-019f5d8b912d admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.709290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a19f2-270d-4553-b687-019f5d8b912d admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.712714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a19f2-270d-4553-b687-019f5d8b912d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:46.712714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a19f2-270d-4553-b687-019f5d8b912d admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:46.724286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-0949568a-f40a-403c-b47b-73f4c3513ba0 None None] MFA Rules not processed for user `687d0375abd44d2481f988c6df90fad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:46.726546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0949568a-f40a-403c-b47b-73f4c3513ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:46.740869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 807/1625] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Aug 12 12:18:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:46.743422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949568a-f40a-403c-b47b-73f4c3513ba0 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:46.743894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0949568a-f40a-403c-b47b-73f4c3513ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:46.758031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949568a-f40a-403c-b47b-73f4c3513ba0 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:46.763105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54442f-7b8f-4734-9ed2-0d1c26e470f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:46.766561 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54442f-7b8f-4734-9ed2-0d1c26e470f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.766561 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54442f-7b8f-4734-9ed2-0d1c26e470f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.766561 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54442f-7b8f-4734-9ed2-0d1c26e470f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.777911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54442f-7b8f-4734-9ed2-0d1c26e470f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:46.779866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54442f-7b8f-4734-9ed2-0d1c26e470f4 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:46.793522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 808/1626] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:18:46.794336 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54442f-7b8f-4734-9ed2-0d1c26e470f4 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:46.804891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54442f-7b8f-4734-9ed2-0d1c26e470f4 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:46.810266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840daad3-ba60-4078-a74c-ce11023d172c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:46.811275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840daad3-ba60-4078-a74c-ce11023d172c service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.811788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840daad3-ba60-4078-a74c-ce11023d172c service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.812160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840daad3-ba60-4078-a74c-ce11023d172c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.825489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840daad3-ba60-4078-a74c-ce11023d172c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:46.827451 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840daad3-ba60-4078-a74c-ce11023d172c service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:46.836694 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 819/1627] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:18:46 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:46.841140 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a19f2-270d-4553-b687-019f5d8b912d admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:46.845705 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 809/1628] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:18:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:18:46.923265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a19f2-270d-4553-b687-019f5d8b912d admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:46.930554 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f92636-f0ef-40de-af4a-8d04295ae008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:46.931849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f92636-f0ef-40de-af4a-8d04295ae008 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:46.932192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f92636-f0ef-40de-af4a-8d04295ae008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:46.932529 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f92636-f0ef-40de-af4a-8d04295ae008 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:46.933377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f92636-f0ef-40de-af4a-8d04295ae008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:46.935655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f92636-f0ef-40de-af4a-8d04295ae008 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:47.043787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 820/1629] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:18:46 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:47.045873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840daad3-ba60-4078-a74c-ce11023d172c service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:47.064539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840daad3-ba60-4078-a74c-ce11023d172c service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:47.071383 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48faa4ff-e389-4cd7-9241-6dfac4177033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:47.072297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48faa4ff-e389-4cd7-9241-6dfac4177033 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:47.073224 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48faa4ff-e389-4cd7-9241-6dfac4177033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:47.073495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48faa4ff-e389-4cd7-9241-6dfac4177033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:47.074450 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48faa4ff-e389-4cd7-9241-6dfac4177033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:47.076228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48faa4ff-e389-4cd7-9241-6dfac4177033 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:47.094426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 810/1630] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:47.103775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f92636-f0ef-40de-af4a-8d04295ae008 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:47.109235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f92636-f0ef-40de-af4a-8d04295ae008 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:47.116353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef9c4a-b366-4d64-9bda-8931ee714498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:47.117751 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef9c4a-b366-4d64-9bda-8931ee714498 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:47.118264 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef9c4a-b366-4d64-9bda-8931ee714498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:47.118920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ef9c4a-b366-4d64-9bda-8931ee714498 admin admin] PATH_INFO: `/v3/projects/8ae584646651473f9cabd9309808115a/users/163e176baa1e479ba03f94d0e5e422c3/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:47.139525 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ef9c4a-b366-4d64-9bda-8931ee714498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae584646651473f9cabd9309808115a, user_id=163e176baa1e479ba03f94d0e5e422c3, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:47.143105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ef9c4a-b366-4d64-9bda-8931ee714498 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:47.236917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 821/1631] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:18:47 2022] PUT /identity/v3/projects/8ae584646651473f9cabd9309808115a/users/163e176baa1e479ba03f94d0e5e422c3/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:18:47.241225 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48faa4ff-e389-4cd7-9241-6dfac4177033 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:47.258939 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48faa4ff-e389-4cd7-9241-6dfac4177033 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:47.390383 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c643e3-9ebd-48c4-9f95-6958ae064806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:47.391001 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c643e3-9ebd-48c4-9f95-6958ae064806 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:47.391283 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c643e3-9ebd-48c4-9f95-6958ae064806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:47.391668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c643e3-9ebd-48c4-9f95-6958ae064806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:47.392347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c643e3-9ebd-48c4-9f95-6958ae064806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:47.394071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c643e3-9ebd-48c4-9f95-6958ae064806 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:47.428712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 811/1632] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:47.430484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef9c4a-b366-4d64-9bda-8931ee714498 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:47.438259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:47.443626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ef9c4a-b366-4d64-9bda-8931ee714498 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:47.450224 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d3f0a-144b-45c6-a533-eec22ea1450f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:47.451269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d3f0a-144b-45c6-a533-eec22ea1450f admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:47.451607 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d3f0a-144b-45c6-a533-eec22ea1450f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:47.451970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d3f0a-144b-45c6-a533-eec22ea1450f admin admin] PATH_INFO: `/v3/projects/8ae584646651473f9cabd9309808115a/users/163e176baa1e479ba03f94d0e5e422c3/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:47.460524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d3f0a-144b-45c6-a533-eec22ea1450f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae584646651473f9cabd9309808115a, user_id=163e176baa1e479ba03f94d0e5e422c3, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:47.460524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d3f0a-144b-45c6-a533-eec22ea1450f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:47.482448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 822/1633] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:18:47 2022] PUT /identity/v3/projects/8ae584646651473f9cabd9309808115a/users/163e176baa1e479ba03f94d0e5e422c3/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:47.484276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4d3a2-7e7d-4769-98d0-a0a76e4c8430 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:47.484570 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4d3a2-7e7d-4769-98d0-a0a76e4c8430 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:47.484910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4d3a2-7e7d-4769-98d0-a0a76e4c8430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:47.537426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-2da4d3a2-7e7d-4769-98d0-a0a76e4c8430 None None] MFA Rules not processed for user `163e176baa1e479ba03f94d0e5e422c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:47.539229 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2da4d3a2-7e7d-4769-98d0-a0a76e4c8430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:47.661140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 812/1634] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:47 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:47.705019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d3f0a-144b-45c6-a533-eec22ea1450f admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:47.769210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d3f0a-144b-45c6-a533-eec22ea1450f admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:47.774039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21ae95-e66a-4928-897f-ce718ede2277 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:47.775162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21ae95-e66a-4928-897f-ce718ede2277 service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:47.775162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21ae95-e66a-4928-897f-ce718ede2277 service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:47.775322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21ae95-e66a-4928-897f-ce718ede2277 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:47.889347 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21ae95-e66a-4928-897f-ce718ede2277 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:47.891076 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21ae95-e66a-4928-897f-ce718ede2277 service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:47.912082 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 823/1635] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:18:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:18:47.967375 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4d3a2-7e7d-4769-98d0-a0a76e4c8430 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:47.982678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4d3a2-7e7d-4769-98d0-a0a76e4c8430 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:47.986661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2fd70-fe2d-40fa-a38b-958ba25ec27d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:47.987676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2fd70-fe2d-40fa-a38b-958ba25ec27d service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:47.987957 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2fd70-fe2d-40fa-a38b-958ba25ec27d service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:47.988306 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2fd70-fe2d-40fa-a38b-958ba25ec27d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:48.010031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2fd70-fe2d-40fa-a38b-958ba25ec27d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:48.011475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2fd70-fe2d-40fa-a38b-958ba25ec27d service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:48.032940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 813/1636] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:18:47 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:18:48.245142 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21ae95-e66a-4928-897f-ce718ede2277 service cinder] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:48.347442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2fd70-fe2d-40fa-a38b-958ba25ec27d service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:48.359054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21ae95-e66a-4928-897f-ce718ede2277 service cinder] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:48.364343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2fd70-fe2d-40fa-a38b-958ba25ec27d service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:48.368343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c23e3-5cf9-4458-86cb-cebd84a9b0d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:48.369017 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c23e3-5cf9-4458-86cb-cebd84a9b0d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:48.369300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c23e3-5cf9-4458-86cb-cebd84a9b0d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:48.369585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c23e3-5cf9-4458-86cb-cebd84a9b0d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:48.370656 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159d1ff-faf8-471d-b432-990ded780867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:48.371910 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159d1ff-faf8-471d-b432-990ded780867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:48.372434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159d1ff-faf8-471d-b432-990ded780867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:48.373191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159d1ff-faf8-471d-b432-990ded780867 admin admin] PATH_INFO: `/v3/users/0e30ba9436f940188d7fed18c8ca9dae` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:48.376178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159d1ff-faf8-471d-b432-990ded780867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e30ba9436f940188d7fed18c8ca9dae)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:48.378822 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: WARNING py.warnings [None req-2159d1ff-faf8-471d-b432-990ded780867 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 12:18:48.378822 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: warnings.warn(msg) Aug 12 12:18:48.378822 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: Aug 12 12:18:48.381166 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159d1ff-faf8-471d-b432-990ded780867 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:48.388948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:48.504499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c23e3-5cf9-4458-86cb-cebd84a9b0d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:48.506669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c23e3-5cf9-4458-86cb-cebd84a9b0d5 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:48.526389 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 814/1637] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:18:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:18:48.708364 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:48.722265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2159d1ff-faf8-471d-b432-990ded780867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e30ba9436f940188d7fed18c8ca9dae'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:48.823046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2159d1ff-faf8-471d-b432-990ded780867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e30ba9436f940188d7fed18c8ca9dae'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:48.824285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2159d1ff-faf8-471d-b432-990ded780867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e30ba9436f940188d7fed18c8ca9dae'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:48.844195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2159d1ff-faf8-471d-b432-990ded780867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e30ba9436f940188d7fed18c8ca9dae'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:48.882870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2159d1ff-faf8-471d-b432-990ded780867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e30ba9436f940188d7fed18c8ca9dae'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:49.175323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 824/1638] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:18:48 2022] DELETE /identity/v3/users/0e30ba9436f940188d7fed18c8ca9dae => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:49.176928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f15109-ecfc-401c-948e-23d43f8b1536 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:49.176928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f15109-ecfc-401c-948e-23d43f8b1536 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:49.176928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f15109-ecfc-401c-948e-23d43f8b1536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:49.176928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-83f15109-ecfc-401c-948e-23d43f8b1536 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:49.176928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-83f15109-ecfc-401c-948e-23d43f8b1536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:49.176928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 815/1639] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:49.178058 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159d1ff-faf8-471d-b432-990ded780867 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:49.178058 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2159d1ff-faf8-471d-b432-990ded780867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:49.178058 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159d1ff-faf8-471d-b432-990ded780867 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:49.178788 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fc30e-a11d-486b-a4f3-31cb79fa7011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:49.178788 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fc30e-a11d-486b-a4f3-31cb79fa7011 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:49.178788 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fc30e-a11d-486b-a4f3-31cb79fa7011 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:49.178788 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fc30e-a11d-486b-a4f3-31cb79fa7011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:49.179410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fc30e-a11d-486b-a4f3-31cb79fa7011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:49.179410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fc30e-a11d-486b-a4f3-31cb79fa7011 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:49.188481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 825/1640] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:18:50.784403 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f15109-ecfc-401c-948e-23d43f8b1536 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:50.786901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-83f15109-ecfc-401c-948e-23d43f8b1536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:50.811884 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f15109-ecfc-401c-948e-23d43f8b1536 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:50.823086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29503467-89c0-4ebc-b82d-5639a19d449d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:50.823744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29503467-89c0-4ebc-b82d-5639a19d449d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:50.824031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29503467-89c0-4ebc-b82d-5639a19d449d admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:50.824308 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29503467-89c0-4ebc-b82d-5639a19d449d admin admin] PATH_INFO: `/v3/projects/9ad47eec693748db9c3c0303fc8cc747` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:50.827206 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29503467-89c0-4ebc-b82d-5639a19d449d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ad47eec693748db9c3c0303fc8cc747)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:50.828278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: WARNING py.warnings [None req-29503467-89c0-4ebc-b82d-5639a19d449d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 12 12:18:50.828278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: warnings.warn(msg) Aug 12 12:18:50.828278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: Aug 12 12:18:50.829531 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29503467-89c0-4ebc-b82d-5639a19d449d admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:51.103035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-29503467-89c0-4ebc-b82d-5639a19d449d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ad47eec693748db9c3c0303fc8cc747', 'request_id': 'req-29503467-89c0-4ebc-b82d-5639a19d449d'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:51.122122 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-29503467-89c0-4ebc-b82d-5639a19d449d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ad47eec693748db9c3c0303fc8cc747', 'request_id': 'req-29503467-89c0-4ebc-b82d-5639a19d449d'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:51.165795 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-29503467-89c0-4ebc-b82d-5639a19d449d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ad47eec693748db9c3c0303fc8cc747', 'request_id': 'req-29503467-89c0-4ebc-b82d-5639a19d449d'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:51.341229 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-29503467-89c0-4ebc-b82d-5639a19d449d admin admin] The token cache is being invalidate because project 9ad47eec693748db9c3c0303fc8cc747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:18:51.341528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-29503467-89c0-4ebc-b82d-5639a19d449d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:51.342209 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-29503467-89c0-4ebc-b82d-5639a19d449d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:51.343753 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 816/1641] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:18:50 2022] DELETE /identity/v3/projects/9ad47eec693748db9c3c0303fc8cc747 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:51.349875 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc0d3b-3be1-482f-a347-ae74303f8fdd None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:51.350376 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc0d3b-3be1-482f-a347-ae74303f8fdd None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:51.350441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc0d3b-3be1-482f-a347-ae74303f8fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:51.382239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-24cc0d3b-3be1-482f-a347-ae74303f8fdd None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:51.389342 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-24cc0d3b-3be1-482f-a347-ae74303f8fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:51.494390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 826/1642] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:51.502769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6810e4-4c56-46f6-ba9d-99ca0a9247c4 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:51.502934 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6810e4-4c56-46f6-ba9d-99ca0a9247c4 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:51.503363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6810e4-4c56-46f6-ba9d-99ca0a9247c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:51.541801 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-cb6810e4-4c56-46f6-ba9d-99ca0a9247c4 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:51.543337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-cb6810e4-4c56-46f6-ba9d-99ca0a9247c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:51.561504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 817/1643] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:51.562246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc0d3b-3be1-482f-a347-ae74303f8fdd None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:51.578546 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc0d3b-3be1-482f-a347-ae74303f8fdd None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:51.592170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001fb7c-f629-433f-8825-ff2a3ecd0c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:51.592824 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001fb7c-f629-433f-8825-ff2a3ecd0c51 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:51.593102 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001fb7c-f629-433f-8825-ff2a3ecd0c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:51.593414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001fb7c-f629-433f-8825-ff2a3ecd0c51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:51.594027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001fb7c-f629-433f-8825-ff2a3ecd0c51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:18:51.594557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001fb7c-f629-433f-8825-ff2a3ecd0c51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:51.597753 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001fb7c-f629-433f-8825-ff2a3ecd0c51 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:51.665099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 827/1644] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:18:51.668917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6810e4-4c56-46f6-ba9d-99ca0a9247c4 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:51.686076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6810e4-4c56-46f6-ba9d-99ca0a9247c4 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:51.710945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99751903-eeb1-44e3-9eae-a8e035f2ce51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:51.711584 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99751903-eeb1-44e3-9eae-a8e035f2ce51 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:51.756860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99751903-eeb1-44e3-9eae-a8e035f2ce51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:51.758311 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99751903-eeb1-44e3-9eae-a8e035f2ce51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:51.760286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99751903-eeb1-44e3-9eae-a8e035f2ce51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:51.763335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99751903-eeb1-44e3-9eae-a8e035f2ce51 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:51.930304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 818/1645] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:18:51 2022] POST /identity/v3/projects => generated 367 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:51.930933 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001fb7c-f629-433f-8825-ff2a3ecd0c51 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:51.952048 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001fb7c-f629-433f-8825-ff2a3ecd0c51 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:52.062776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:52.123229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9f148-cad8-42bc-81ec-f915e2c8ffe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:52.123614 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9f148-cad8-42bc-81ec-f915e2c8ffe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:52.123917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9f148-cad8-42bc-81ec-f915e2c8ffe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:52.124179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9f148-cad8-42bc-81ec-f915e2c8ffe5 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:52.127907 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9f148-cad8-42bc-81ec-f915e2c8ffe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:52.133452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9f148-cad8-42bc-81ec-f915e2c8ffe5 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:52.290895 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 828/1646] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:18:51 2022] POST /identity/v3/users => generated 284 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:18:52.294769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99751903-eeb1-44e3-9eae-a8e035f2ce51 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:52.319328 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99751903-eeb1-44e3-9eae-a8e035f2ce51 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:52.341691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a94c7-288c-448a-b84b-09c2f206bc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:52.351246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a94c7-288c-448a-b84b-09c2f206bc48 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:52.351690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a94c7-288c-448a-b84b-09c2f206bc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:52.353574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a94c7-288c-448a-b84b-09c2f206bc48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:52.355381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a94c7-288c-448a-b84b-09c2f206bc48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:52.359494 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a94c7-288c-448a-b84b-09c2f206bc48 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:52.367085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:52.379891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 819/1647] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:52.384461 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9f148-cad8-42bc-81ec-f915e2c8ffe5 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:52.403263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9f148-cad8-42bc-81ec-f915e2c8ffe5 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:52.408948 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b820c-0819-44a8-a5ed-4a30c508848f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:52.423566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b820c-0819-44a8-a5ed-4a30c508848f admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:52.427615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b820c-0819-44a8-a5ed-4a30c508848f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:52.428428 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b820c-0819-44a8-a5ed-4a30c508848f admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:52.436604 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b820c-0819-44a8-a5ed-4a30c508848f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:52.436604 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b820c-0819-44a8-a5ed-4a30c508848f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:52.459965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 829/1648] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:52.463030 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a94c7-288c-448a-b84b-09c2f206bc48 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:52.490514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a94c7-288c-448a-b84b-09c2f206bc48 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:52.521419 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d286f0-08f6-4bc8-b42c-e010eaf5d1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:52.522128 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d286f0-08f6-4bc8-b42c-e010eaf5d1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:52.522414 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d286f0-08f6-4bc8-b42c-e010eaf5d1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:52.522698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d286f0-08f6-4bc8-b42c-e010eaf5d1cf admin admin] PATH_INFO: `/v3/projects/f95d088636e74674b2c3e5bd7e04129b/users/bf9e6de471fb45e19e35dfff970e8a62/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:52.579233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d286f0-08f6-4bc8-b42c-e010eaf5d1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f95d088636e74674b2c3e5bd7e04129b, user_id=bf9e6de471fb45e19e35dfff970e8a62, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:52.582045 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d286f0-08f6-4bc8-b42c-e010eaf5d1cf admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:52.652716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 820/1649] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:18:52 2022] PUT /identity/v3/projects/f95d088636e74674b2c3e5bd7e04129b/users/bf9e6de471fb45e19e35dfff970e8a62/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:52.664869 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dc291-41fe-4e1e-a0f5-2faf6a71b45b None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:52.681314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dc291-41fe-4e1e-a0f5-2faf6a71b45b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:52.683886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dc291-41fe-4e1e-a0f5-2faf6a71b45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:52.737409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-593dc291-41fe-4e1e-a0f5-2faf6a71b45b None None] MFA Rules not processed for user `bf9e6de471fb45e19e35dfff970e8a62`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:52.742803 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-593dc291-41fe-4e1e-a0f5-2faf6a71b45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:52.975592 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 830/1650] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:52.980451 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d286f0-08f6-4bc8-b42c-e010eaf5d1cf admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:53.122892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d286f0-08f6-4bc8-b42c-e010eaf5d1cf admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:53.136353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e82b9-250d-4b19-b6a3-e2c8ff685bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:53.137018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e82b9-250d-4b19-b6a3-e2c8ff685bff admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:53.138341 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e82b9-250d-4b19-b6a3-e2c8ff685bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:53.138635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e82b9-250d-4b19-b6a3-e2c8ff685bff admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:53.145901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e82b9-250d-4b19-b6a3-e2c8ff685bff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:53.145901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e82b9-250d-4b19-b6a3-e2c8ff685bff admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:53.289075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 821/1651] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:18:52 2022] POST /identity/v3/projects => generated 367 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:53.293062 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dc291-41fe-4e1e-a0f5-2faf6a71b45b None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:53.315072 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dc291-41fe-4e1e-a0f5-2faf6a71b45b None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:53.400052 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca55dae-a13f-4946-a57f-aeef0f0f8413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:53.401506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca55dae-a13f-4946-a57f-aeef0f0f8413 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:53.401925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca55dae-a13f-4946-a57f-aeef0f0f8413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:53.402578 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca55dae-a13f-4946-a57f-aeef0f0f8413 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:53.403364 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca55dae-a13f-4946-a57f-aeef0f0f8413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:53.405902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca55dae-a13f-4946-a57f-aeef0f0f8413 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:53.534800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 831/1652] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:18:53 2022] POST /identity/v3/users => generated 290 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:53.538461 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e82b9-250d-4b19-b6a3-e2c8ff685bff admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:53.555684 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e82b9-250d-4b19-b6a3-e2c8ff685bff admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:53.570449 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c3059-0e1a-4299-9776-722db650af5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:53.571189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c3059-0e1a-4299-9776-722db650af5b admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:53.572009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c3059-0e1a-4299-9776-722db650af5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:53.574204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c3059-0e1a-4299-9776-722db650af5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:53.574984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c3059-0e1a-4299-9776-722db650af5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:53.580751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c3059-0e1a-4299-9776-722db650af5b admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:53.602719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 822/1653] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:53.605529 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca55dae-a13f-4946-a57f-aeef0f0f8413 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:53.618674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca55dae-a13f-4946-a57f-aeef0f0f8413 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:53.636139 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c3771-68d8-4e5e-b597-a4ebfe9196bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:53.637233 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c3771-68d8-4e5e-b597-a4ebfe9196bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:53.637631 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c3771-68d8-4e5e-b597-a4ebfe9196bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:53.637980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c3771-68d8-4e5e-b597-a4ebfe9196bb admin admin] PATH_INFO: `/v3/projects/ba0ab342961d4cb8a7bf2bf76ad7eb2e/users/ee95d39e96284d5f9a99c5a45fd0259d/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:53.667647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c3771-68d8-4e5e-b597-a4ebfe9196bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0ab342961d4cb8a7bf2bf76ad7eb2e, user_id=ee95d39e96284d5f9a99c5a45fd0259d, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:53.670808 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c3771-68d8-4e5e-b597-a4ebfe9196bb admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:53.776914 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 832/1654] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:18:53 2022] PUT /identity/v3/projects/ba0ab342961d4cb8a7bf2bf76ad7eb2e/users/ee95d39e96284d5f9a99c5a45fd0259d/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:53.781079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c3059-0e1a-4299-9776-722db650af5b admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:53.797229 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c3059-0e1a-4299-9776-722db650af5b admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:53.918614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324471e-38ba-4ed0-8fdd-11d8a3dc6443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:53.919373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324471e-38ba-4ed0-8fdd-11d8a3dc6443 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:53.919780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324471e-38ba-4ed0-8fdd-11d8a3dc6443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:53.920154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324471e-38ba-4ed0-8fdd-11d8a3dc6443 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:53.920915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324471e-38ba-4ed0-8fdd-11d8a3dc6443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:53.922619 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324471e-38ba-4ed0-8fdd-11d8a3dc6443 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:53.942987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 823/1655] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:53.947557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c3771-68d8-4e5e-b597-a4ebfe9196bb admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:53.972867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c3771-68d8-4e5e-b597-a4ebfe9196bb admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:53.979562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f1d5c-b0f9-4610-a796-a3f04ed35374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:53.981029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f1d5c-b0f9-4610-a796-a3f04ed35374 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:53.981543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f1d5c-b0f9-4610-a796-a3f04ed35374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:53.982066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f1d5c-b0f9-4610-a796-a3f04ed35374 admin admin] PATH_INFO: `/v3/projects/ba0ab342961d4cb8a7bf2bf76ad7eb2e/users/ee95d39e96284d5f9a99c5a45fd0259d/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:53.985141 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f1d5c-b0f9-4610-a796-a3f04ed35374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0ab342961d4cb8a7bf2bf76ad7eb2e, user_id=ee95d39e96284d5f9a99c5a45fd0259d, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:53.987830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f1d5c-b0f9-4610-a796-a3f04ed35374 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:54.024899 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 833/1656] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:18:53 2022] PUT /identity/v3/projects/ba0ab342961d4cb8a7bf2bf76ad7eb2e/users/ee95d39e96284d5f9a99c5a45fd0259d/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:54.037041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057a5d8-86cb-4da0-b57f-282927fb52ae None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:54.037564 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057a5d8-86cb-4da0-b57f-282927fb52ae None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:54.037850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057a5d8-86cb-4da0-b57f-282927fb52ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:54.130432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-b057a5d8-86cb-4da0-b57f-282927fb52ae None None] MFA Rules not processed for user `ee95d39e96284d5f9a99c5a45fd0259d`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:54.131807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b057a5d8-86cb-4da0-b57f-282927fb52ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:54.276356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:54.392460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 824/1657] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:54 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:54.421455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f1d5c-b0f9-4610-a796-a3f04ed35374 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:54.423540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-215f1d5c-b0f9-4610-a796-a3f04ed35374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:54.430882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:54.606935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f1d5c-b0f9-4610-a796-a3f04ed35374 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:54.620312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d64ef0-7e08-44ba-9954-639d491f4845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:54.622457 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d64ef0-7e08-44ba-9954-639d491f4845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:54.630451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d64ef0-7e08-44ba-9954-639d491f4845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:54.630451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d64ef0-7e08-44ba-9954-639d491f4845 admin admin] PATH_INFO: `/v3/users/253488caedb44ea4b3b7c1c8f5dc8731` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:54.630451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d64ef0-7e08-44ba-9954-639d491f4845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=253488caedb44ea4b3b7c1c8f5dc8731)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:54.630853 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d64ef0-7e08-44ba-9954-639d491f4845 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:54.961801 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-94d64ef0-7e08-44ba-9954-639d491f4845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253488caedb44ea4b3b7c1c8f5dc8731'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:55.048866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057a5d8-86cb-4da0-b57f-282927fb52ae None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:55.050184 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-94d64ef0-7e08-44ba-9954-639d491f4845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253488caedb44ea4b3b7c1c8f5dc8731'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:55.051050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-94d64ef0-7e08-44ba-9954-639d491f4845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253488caedb44ea4b3b7c1c8f5dc8731'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:55.051832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b057a5d8-86cb-4da0-b57f-282927fb52ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:55.062116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-94d64ef0-7e08-44ba-9954-639d491f4845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253488caedb44ea4b3b7c1c8f5dc8731'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:55.104416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-94d64ef0-7e08-44ba-9954-639d491f4845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253488caedb44ea4b3b7c1c8f5dc8731'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:55.117967 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 834/1658] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:18:54 2022] DELETE /identity/v3/users/253488caedb44ea4b3b7c1c8f5dc8731 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:55.122748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af71d77-1487-4ed1-97c1-05d9474d3bf1 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:55.123043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af71d77-1487-4ed1-97c1-05d9474d3bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:55.123380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af71d77-1487-4ed1-97c1-05d9474d3bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:55.171223 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9af71d77-1487-4ed1-97c1-05d9474d3bf1 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:55.177121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9af71d77-1487-4ed1-97c1-05d9474d3bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:55.187889 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057a5d8-86cb-4da0-b57f-282927fb52ae None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:55.302434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa0702-f9fc-408d-b7aa-dd970be70fdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:55.303264 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa0702-f9fc-408d-b7aa-dd970be70fdf service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:55.305624 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa0702-f9fc-408d-b7aa-dd970be70fdf service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:55.305624 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa0702-f9fc-408d-b7aa-dd970be70fdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:55.312296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4eaa0702-f9fc-408d-b7aa-dd970be70fdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:55.325385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 835/1659] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:55.377050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af71d77-1487-4ed1-97c1-05d9474d3bf1 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:55.377050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9af71d77-1487-4ed1-97c1-05d9474d3bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:55.451172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af71d77-1487-4ed1-97c1-05d9474d3bf1 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:55.460090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f57101-4b49-4ab7-8f17-b3c8a26a33cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:55.460923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f57101-4b49-4ab7-8f17-b3c8a26a33cf service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:55.461311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f57101-4b49-4ab7-8f17-b3c8a26a33cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:55.461689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f57101-4b49-4ab7-8f17-b3c8a26a33cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:55.472007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f57101-4b49-4ab7-8f17-b3c8a26a33cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:55.473643 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f57101-4b49-4ab7-8f17-b3c8a26a33cf service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:55.485333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 836/1660] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:18:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:18:55.520051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa0702-f9fc-408d-b7aa-dd970be70fdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:55.523422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa0702-f9fc-408d-b7aa-dd970be70fdf service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:55.570664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 825/1661] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:18:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:18:56.185798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:56.728677 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f57101-4b49-4ab7-8f17-b3c8a26a33cf service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:56.730324 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-84f57101-4b49-4ab7-8f17-b3c8a26a33cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:56.754523 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f57101-4b49-4ab7-8f17-b3c8a26a33cf service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:56.767222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c16ed-cea3-4e0c-ac86-509e83a7a6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:56.767939 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c16ed-cea3-4e0c-ac86-509e83a7a6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:56.768229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c16ed-cea3-4e0c-ac86-509e83a7a6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:56.768506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c16ed-cea3-4e0c-ac86-509e83a7a6cf admin admin] PATH_INFO: `/v3/projects/29877b59841f428f9d094a309fdce828` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:56.771088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c16ed-cea3-4e0c-ac86-509e83a7a6cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=29877b59841f428f9d094a309fdce828)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:56.775322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c16ed-cea3-4e0c-ac86-509e83a7a6cf admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:56.977605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ea5c16ed-cea3-4e0c-ac86-509e83a7a6cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29877b59841f428f9d094a309fdce828', 'request_id': 'req-ea5c16ed-cea3-4e0c-ac86-509e83a7a6cf'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:56.994639 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ea5c16ed-cea3-4e0c-ac86-509e83a7a6cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29877b59841f428f9d094a309fdce828', 'request_id': 'req-ea5c16ed-cea3-4e0c-ac86-509e83a7a6cf'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:57.021390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ea5c16ed-cea3-4e0c-ac86-509e83a7a6cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29877b59841f428f9d094a309fdce828', 'request_id': 'req-ea5c16ed-cea3-4e0c-ac86-509e83a7a6cf'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:57.158232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ea5c16ed-cea3-4e0c-ac86-509e83a7a6cf admin admin] The token cache is being invalidate because project 29877b59841f428f9d094a309fdce828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:18:57.158522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ea5c16ed-cea3-4e0c-ac86-509e83a7a6cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:57.169525 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ea5c16ed-cea3-4e0c-ac86-509e83a7a6cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:18:57.183040 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 837/1662] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:18:56 2022] DELETE /identity/v3/projects/29877b59841f428f9d094a309fdce828 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:57.184577 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67d02d-d76b-4cc8-9d29-e983a5c11861 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:57.185069 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67d02d-d76b-4cc8-9d29-e983a5c11861 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:57.185267 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67d02d-d76b-4cc8-9d29-e983a5c11861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:57.349174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-fa67d02d-d76b-4cc8-9d29-e983a5c11861 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:57.361904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-fa67d02d-d76b-4cc8-9d29-e983a5c11861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:57.540044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:57.619348 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 826/1663] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:57.620175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cde340-c894-485d-a3dd-fce581f70f6c None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:57.620573 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cde340-c894-485d-a3dd-fce581f70f6c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:57.620986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cde340-c894-485d-a3dd-fce581f70f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:57.683747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f5cde340-c894-485d-a3dd-fce581f70f6c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:57.688397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f5cde340-c894-485d-a3dd-fce581f70f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:57.717679 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 838/1664] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:57.719517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67d02d-d76b-4cc8-9d29-e983a5c11861 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:57.733783 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67d02d-d76b-4cc8-9d29-e983a5c11861 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:57.752805 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ffdf9-cdfa-4286-9d2e-ad7ad3938cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:57.754846 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ffdf9-cdfa-4286-9d2e-ad7ad3938cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:57.755143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ffdf9-cdfa-4286-9d2e-ad7ad3938cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:57.755911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ffdf9-cdfa-4286-9d2e-ad7ad3938cc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:57.756837 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ffdf9-cdfa-4286-9d2e-ad7ad3938cc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:18:57.760865 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ffdf9-cdfa-4286-9d2e-ad7ad3938cc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:57.760865 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ffdf9-cdfa-4286-9d2e-ad7ad3938cc1 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:57.790450 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 827/1665] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:18:57.795748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cde340-c894-485d-a3dd-fce581f70f6c None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:57.817269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cde340-c894-485d-a3dd-fce581f70f6c None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:57.833820 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda20ff2-4786-4e9b-be4f-1ea265b3341f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:57.836952 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda20ff2-4786-4e9b-be4f-1ea265b3341f admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:57.837386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda20ff2-4786-4e9b-be4f-1ea265b3341f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:57.838182 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda20ff2-4786-4e9b-be4f-1ea265b3341f admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:57.841140 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda20ff2-4786-4e9b-be4f-1ea265b3341f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:57.843670 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda20ff2-4786-4e9b-be4f-1ea265b3341f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:57.967199 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 839/1666] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:18:57 2022] POST /identity/v3/projects => generated 409 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:57.973496 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ffdf9-cdfa-4286-9d2e-ad7ad3938cc1 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:57.989286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ffdf9-cdfa-4286-9d2e-ad7ad3938cc1 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:58.109324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b999e-1ef2-48f9-9ed0-f2ed8f8c7e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:58.110295 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b999e-1ef2-48f9-9ed0-f2ed8f8c7e91 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:58.110767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b999e-1ef2-48f9-9ed0-f2ed8f8c7e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:58.111143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b999e-1ef2-48f9-9ed0-f2ed8f8c7e91 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:58.111937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b999e-1ef2-48f9-9ed0-f2ed8f8c7e91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:58.115364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b999e-1ef2-48f9-9ed0-f2ed8f8c7e91 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:58.224011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 828/1667] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:18:57 2022] POST /identity/v3/users => generated 305 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:58.225256 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda20ff2-4786-4e9b-be4f-1ea265b3341f admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:58.248674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda20ff2-4786-4e9b-be4f-1ea265b3341f admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:58.256102 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a9962-7fb0-416a-92d1-3d69ff7d3052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:58.256839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a9962-7fb0-416a-92d1-3d69ff7d3052 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:58.257131 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a9962-7fb0-416a-92d1-3d69ff7d3052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:58.257414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a9962-7fb0-416a-92d1-3d69ff7d3052 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:58.258114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a9962-7fb0-416a-92d1-3d69ff7d3052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:58.262262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a9962-7fb0-416a-92d1-3d69ff7d3052 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:58.292547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 840/1668] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:58.294744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b999e-1ef2-48f9-9ed0-f2ed8f8c7e91 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:58.309972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b999e-1ef2-48f9-9ed0-f2ed8f8c7e91 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:58.325264 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b84269-d917-4927-8c47-ba7c491e158d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:58.326075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b84269-d917-4927-8c47-ba7c491e158d admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:58.327096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b84269-d917-4927-8c47-ba7c491e158d admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:58.327548 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b84269-d917-4927-8c47-ba7c491e158d admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:58.328401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b84269-d917-4927-8c47-ba7c491e158d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:58.330907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b84269-d917-4927-8c47-ba7c491e158d admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:58.359675 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 829/1669] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:58.365003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a9962-7fb0-416a-92d1-3d69ff7d3052 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:58.400099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a9962-7fb0-416a-92d1-3d69ff7d3052 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:58.426696 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7203c0-3211-495c-8e52-279eb45030d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:58.428347 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7203c0-3211-495c-8e52-279eb45030d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:58.433473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7203c0-3211-495c-8e52-279eb45030d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:58.434195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7203c0-3211-495c-8e52-279eb45030d7 admin admin] PATH_INFO: `/v3/projects/bacf73e4c9714e7daa0b61af79f767c5/users/2710c46896c642dca92b3cfe2ea4bf8f/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:58.483120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7203c0-3211-495c-8e52-279eb45030d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bacf73e4c9714e7daa0b61af79f767c5, user_id=2710c46896c642dca92b3cfe2ea4bf8f, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:58.489709 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7203c0-3211-495c-8e52-279eb45030d7 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:58.688973 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 841/1670] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:18:58 2022] PUT /identity/v3/projects/bacf73e4c9714e7daa0b61af79f767c5/users/2710c46896c642dca92b3cfe2ea4bf8f/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:58.703506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcae37c-0515-4c10-89b7-dcd1c6306ec6 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:58.703799 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcae37c-0515-4c10-89b7-dcd1c6306ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:58.704088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcae37c-0515-4c10-89b7-dcd1c6306ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:58.780052 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-afcae37c-0515-4c10-89b7-dcd1c6306ec6 None None] MFA Rules not processed for user `2710c46896c642dca92b3cfe2ea4bf8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:58.781304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-afcae37c-0515-4c10-89b7-dcd1c6306ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:58.951880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 830/1671] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:58 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:58.955584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7203c0-3211-495c-8e52-279eb45030d7 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:59.015930 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:59.042117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb684e-28aa-45c0-a5b0-d1195cfe95f0 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:59.042332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb684e-28aa-45c0-a5b0-d1195cfe95f0 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:59.042606 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb684e-28aa-45c0-a5b0-d1195cfe95f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:59.073073 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7203c0-3211-495c-8e52-279eb45030d7 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:59.086063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35beeb0e-0336-487e-8ecc-7dbd7e1a6fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:59.087021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35beeb0e-0336-487e-8ecc-7dbd7e1a6fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:59.087558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35beeb0e-0336-487e-8ecc-7dbd7e1a6fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:59.088031 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35beeb0e-0336-487e-8ecc-7dbd7e1a6fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:59.088150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-6ebb684e-28aa-45c0-a5b0-d1195cfe95f0 None None] MFA Rules not processed for user `9bc7ffa8e3cf49fe9dcd8471c25be83a`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:59.089011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35beeb0e-0336-487e-8ecc-7dbd7e1a6fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:59.089699 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6ebb684e-28aa-45c0-a5b0-d1195cfe95f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:59.097608 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35beeb0e-0336-487e-8ecc-7dbd7e1a6fd2 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:59.125033 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:18:59.224261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 842/1672] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:18:58 2022] POST /identity/v3/projects => generated 407 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:59.228329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35beeb0e-0336-487e-8ecc-7dbd7e1a6fd2 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:59.239835 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 831/1673] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:59.254679 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1ca81-20ce-408b-86b8-bd7a8a5ed179 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:59.256047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1ca81-20ce-408b-86b8-bd7a8a5ed179 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:59.256544 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1ca81-20ce-408b-86b8-bd7a8a5ed179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:59.323308 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-96b1ca81-20ce-408b-86b8-bd7a8a5ed179 None None] MFA Rules not processed for user `9bc7ffa8e3cf49fe9dcd8471c25be83a`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:18:59.326488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-96b1ca81-20ce-408b-86b8-bd7a8a5ed179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:59.374986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35beeb0e-0336-487e-8ecc-7dbd7e1a6fd2 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:59.384830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8fd32-6f4b-420c-9b50-6914af8b08aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:59.385620 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8fd32-6f4b-420c-9b50-6914af8b08aa admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:59.386007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8fd32-6f4b-420c-9b50-6914af8b08aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:59.386385 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8fd32-6f4b-420c-9b50-6914af8b08aa admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:59.387243 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8fd32-6f4b-420c-9b50-6914af8b08aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:59.389765 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8fd32-6f4b-420c-9b50-6914af8b08aa admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:59.469368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 832/1674] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:18:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:18:59.479183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b1ca81-20ce-408b-86b8-bd7a8a5ed179 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:59.480749 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-96b1ca81-20ce-408b-86b8-bd7a8a5ed179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:18:59.533381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 843/1675] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:18:59 2022] POST /identity/v3/users => generated 310 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:18:59.535433 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8fd32-6f4b-420c-9b50-6914af8b08aa admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:59.589605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8fd32-6f4b-420c-9b50-6914af8b08aa admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:59.612764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4553a80-466e-41cf-b0c3-fc99bd0553b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:59.613399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4553a80-466e-41cf-b0c3-fc99bd0553b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:59.613688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4553a80-466e-41cf-b0c3-fc99bd0553b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:59.613967 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4553a80-466e-41cf-b0c3-fc99bd0553b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:59.614632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4553a80-466e-41cf-b0c3-fc99bd0553b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:59.616267 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4553a80-466e-41cf-b0c3-fc99bd0553b3 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:59.629390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 844/1676] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:18:59.634911 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4553a80-466e-41cf-b0c3-fc99bd0553b3 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:59.654538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4553a80-466e-41cf-b0c3-fc99bd0553b3 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:59.671727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2bfee-910c-4348-9b0d-e2c9fb26d0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:59.672517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2bfee-910c-4348-9b0d-e2c9fb26d0fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:59.674329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2bfee-910c-4348-9b0d-e2c9fb26d0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:59.674672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2bfee-910c-4348-9b0d-e2c9fb26d0fb admin admin] PATH_INFO: `/v3/projects/0da08fee56ad48c6932daef85f31a9c5/users/da67d8dc151245158cde6b96c3cfa754/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:59.711238 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb2bfee-910c-4348-9b0d-e2c9fb26d0fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da08fee56ad48c6932daef85f31a9c5, user_id=da67d8dc151245158cde6b96c3cfa754, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:59.716312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb2bfee-910c-4348-9b0d-e2c9fb26d0fb admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:59.740772 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b1ca81-20ce-408b-86b8-bd7a8a5ed179 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:18:59.756587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1c04a-c29e-4900-b9b5-b3c9bb276150 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:18:59.757274 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1c04a-c29e-4900-b9b5-b3c9bb276150 service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:18:59.757563 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1c04a-c29e-4900-b9b5-b3c9bb276150 service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:18:59.757835 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1c04a-c29e-4900-b9b5-b3c9bb276150 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:18:59.783263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1c04a-c29e-4900-b9b5-b3c9bb276150 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:18:59.786304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1c04a-c29e-4900-b9b5-b3c9bb276150 service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:18:59.831854 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 833/1677] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:18:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:18:59.852534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 845/1678] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:18:59 2022] PUT /identity/v3/projects/0da08fee56ad48c6932daef85f31a9c5/users/da67d8dc151245158cde6b96c3cfa754/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:18:59.856522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1c04a-c29e-4900-b9b5-b3c9bb276150 service nova] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:18:59.870910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1c04a-c29e-4900-b9b5-b3c9bb276150 service nova] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:00.021373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce37b8b-ea53-41ac-b9be-4173470c7092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:00.022009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce37b8b-ea53-41ac-b9be-4173470c7092 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:00.022290 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce37b8b-ea53-41ac-b9be-4173470c7092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:00.022565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce37b8b-ea53-41ac-b9be-4173470c7092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:00.023218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce37b8b-ea53-41ac-b9be-4173470c7092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:00.025068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce37b8b-ea53-41ac-b9be-4173470c7092 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:00.060139 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 834/1679] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:00.064136 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2bfee-910c-4348-9b0d-e2c9fb26d0fb admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:00.079355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2bfee-910c-4348-9b0d-e2c9fb26d0fb admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:00.087393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5334e17-3348-4681-8e13-b46dbe7fc84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:00.088112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5334e17-3348-4681-8e13-b46dbe7fc84d admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:00.088402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5334e17-3348-4681-8e13-b46dbe7fc84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:00.088751 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5334e17-3348-4681-8e13-b46dbe7fc84d admin admin] PATH_INFO: `/v3/projects/0da08fee56ad48c6932daef85f31a9c5/users/da67d8dc151245158cde6b96c3cfa754/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:00.093384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5334e17-3348-4681-8e13-b46dbe7fc84d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da08fee56ad48c6932daef85f31a9c5, user_id=da67d8dc151245158cde6b96c3cfa754, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:00.099641 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5334e17-3348-4681-8e13-b46dbe7fc84d admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:00.176893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 846/1680] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:00 2022] PUT /identity/v3/projects/0da08fee56ad48c6932daef85f31a9c5/users/da67d8dc151245158cde6b96c3cfa754/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:00.196947 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584bb3a-4164-4617-9d26-e11757c5f8f7 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:00.197379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584bb3a-4164-4617-9d26-e11757c5f8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:00.197770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584bb3a-4164-4617-9d26-e11757c5f8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:00.242744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-f584bb3a-4164-4617-9d26-e11757c5f8f7 None None] MFA Rules not processed for user `da67d8dc151245158cde6b96c3cfa754`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:00.244004 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f584bb3a-4164-4617-9d26-e11757c5f8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:00.264426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:00.318234 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5334e17-3348-4681-8e13-b46dbe7fc84d admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:00.321348 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f5334e17-3348-4681-8e13-b46dbe7fc84d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:00.434267 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 835/1681] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:00 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:00.443527 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584bb3a-4164-4617-9d26-e11757c5f8f7 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:00.446084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f584bb3a-4164-4617-9d26-e11757c5f8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:00.464070 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5334e17-3348-4681-8e13-b46dbe7fc84d admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:00.479658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a58507-b06d-4819-b7bd-24d12d828c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:00.480491 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a58507-b06d-4819-b7bd-24d12d828c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:00.480976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a58507-b06d-4819-b7bd-24d12d828c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:00.481389 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a58507-b06d-4819-b7bd-24d12d828c92 admin admin] PATH_INFO: `/v3/users/bf9e6de471fb45e19e35dfff970e8a62` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:00.487808 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a58507-b06d-4819-b7bd-24d12d828c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf9e6de471fb45e19e35dfff970e8a62)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:00.492775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a58507-b06d-4819-b7bd-24d12d828c92 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:00.521429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:00.573441 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584bb3a-4164-4617-9d26-e11757c5f8f7 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:00.585190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c280d-6e5b-4400-ab43-51c2ba93132f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:00.585881 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c280d-6e5b-4400-ab43-51c2ba93132f service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:00.586204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c280d-6e5b-4400-ab43-51c2ba93132f service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:00.586515 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c280d-6e5b-4400-ab43-51c2ba93132f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:00.614384 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c280d-6e5b-4400-ab43-51c2ba93132f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:00.615852 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c280d-6e5b-4400-ab43-51c2ba93132f service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:00.642935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 836/1682] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:00 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:00.748996 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed6b97-c547-49c2-b94d-4561dfebd38e None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:00.749370 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed6b97-c547-49c2-b94d-4561dfebd38e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:00.757112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed6b97-c547-49c2-b94d-4561dfebd38e None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:00.760218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 837/1683] 10.222.0.27 () {54 vars in 912 bytes} [Fri Aug 12 12:19:00 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:19:00.765975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dc40f-d800-4ea1-a8d7-093800cbea98 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:00.766445 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dc40f-d800-4ea1-a8d7-093800cbea98 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:00.766732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dc40f-d800-4ea1-a8d7-093800cbea98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:00.782311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-14a58507-b06d-4819-b7bd-24d12d828c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf9e6de471fb45e19e35dfff970e8a62'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:00.840028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-657dc40f-d800-4ea1-a8d7-093800cbea98 None None] MFA Rules not processed for user `745fad9c109a4280906eb0349849117d`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:00.842030 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-657dc40f-d800-4ea1-a8d7-093800cbea98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:00.854728 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-14a58507-b06d-4819-b7bd-24d12d828c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf9e6de471fb45e19e35dfff970e8a62'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:00.855558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-14a58507-b06d-4819-b7bd-24d12d828c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf9e6de471fb45e19e35dfff970e8a62'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:00.879199 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-14a58507-b06d-4819-b7bd-24d12d828c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf9e6de471fb45e19e35dfff970e8a62'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:00.919075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-14a58507-b06d-4819-b7bd-24d12d828c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf9e6de471fb45e19e35dfff970e8a62'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:00.939018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 847/1684] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:00 2022] DELETE /identity/v3/users/bf9e6de471fb45e19e35dfff970e8a62 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:00.943337 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba83b2-1091-4db9-9e6f-b415009b899e None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:00.943627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba83b2-1091-4db9-9e6f-b415009b899e None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:00.943904 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba83b2-1091-4db9-9e6f-b415009b899e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:01.017435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-1fba83b2-1091-4db9-9e6f-b415009b899e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:01.018752 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-1fba83b2-1091-4db9-9e6f-b415009b899e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:01.074200 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 838/1685] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Aug 12 12:19:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:01.080617 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dc40f-d800-4ea1-a8d7-093800cbea98 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:01.083100 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-657dc40f-d800-4ea1-a8d7-093800cbea98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:01.126506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dc40f-d800-4ea1-a8d7-093800cbea98 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:01.142980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:01.207121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 848/1686] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:01.217419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba83b2-1091-4db9-9e6f-b415009b899e None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:01.231870 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa2d0c-469a-47c0-b7e0-41d7b1b0376b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:01.234839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa2d0c-469a-47c0-b7e0-41d7b1b0376b service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:01.235944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa2d0c-469a-47c0-b7e0-41d7b1b0376b service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:01.236506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa2d0c-469a-47c0-b7e0-41d7b1b0376b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:01.257293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa2d0c-469a-47c0-b7e0-41d7b1b0376b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:01.260013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa2d0c-469a-47c0-b7e0-41d7b1b0376b service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:01.283763 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 839/1687] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:19:01.296862 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa2d0c-469a-47c0-b7e0-41d7b1b0376b service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:01.300189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-abaa2d0c-469a-47c0-b7e0-41d7b1b0376b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:01.378936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba83b2-1091-4db9-9e6f-b415009b899e None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:01.390085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e3cce-5041-45b6-9137-40bfda7d9091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:01.390826 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e3cce-5041-45b6-9137-40bfda7d9091 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:01.391109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e3cce-5041-45b6-9137-40bfda7d9091 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:01.391376 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e3cce-5041-45b6-9137-40bfda7d9091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:01.413463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa2d0c-469a-47c0-b7e0-41d7b1b0376b service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:01.427042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2c499-9c28-4c20-91c4-39516744f48e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:01.428052 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2c499-9c28-4c20-91c4-39516744f48e service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:01.428448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e3cce-5041-45b6-9137-40bfda7d9091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:01.428925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2c499-9c28-4c20-91c4-39516744f48e service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:01.429380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2c499-9c28-4c20-91c4-39516744f48e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:01.431468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8fa2c499-9c28-4c20-91c4-39516744f48e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:01.433664 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e3cce-5041-45b6-9137-40bfda7d9091 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:01.466025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 849/1688] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:19:01.507442 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b86da-b44b-47f0-b413-5810357e8a7d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:01.507528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b86da-b44b-47f0-b413-5810357e8a7d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:01.509050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b86da-b44b-47f0-b413-5810357e8a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:01.557989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2c499-9c28-4c20-91c4-39516744f48e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:01.560925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2c499-9c28-4c20-91c4-39516744f48e service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:01.599304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 840/1689] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:01 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:01.619175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-276b86da-b44b-47f0-b413-5810357e8a7d None None] MFA Rules not processed for user `099b87a6c6b8409ab188c0b71ded7f55`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:01.620960 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-276b86da-b44b-47f0-b413-5810357e8a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:01.842658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 850/1690] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:01.854581 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3cb3e-90f2-4be7-94fe-c7922d0d4a75 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:01.854884 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3cb3e-90f2-4be7-94fe-c7922d0d4a75 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:01.855148 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3cb3e-90f2-4be7-94fe-c7922d0d4a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:01.931991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-a9d3cb3e-90f2-4be7-94fe-c7922d0d4a75 None None] MFA Rules not processed for user `099b87a6c6b8409ab188c0b71ded7f55`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:01.933258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a9d3cb3e-90f2-4be7-94fe-c7922d0d4a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:01.966556 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:01.981955 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 841/1691] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:01.990668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b86da-b44b-47f0-b413-5810357e8a7d None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:01.993183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-276b86da-b44b-47f0-b413-5810357e8a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:02.242538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b86da-b44b-47f0-b413-5810357e8a7d None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:02.263404 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acba0d-600a-49fa-9f18-4d0aca462cd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:02.267525 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acba0d-600a-49fa-9f18-4d0aca462cd9 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:02.267974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acba0d-600a-49fa-9f18-4d0aca462cd9 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:02.268449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acba0d-600a-49fa-9f18-4d0aca462cd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:02.295846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acba0d-600a-49fa-9f18-4d0aca462cd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:02.297715 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acba0d-600a-49fa-9f18-4d0aca462cd9 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:02.349834 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 851/1692] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:01 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:02.606778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de627f9-5d16-434d-83eb-98aedb212980 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:02.608490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de627f9-5d16-434d-83eb-98aedb212980 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:02.608997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de627f9-5d16-434d-83eb-98aedb212980 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:02.614589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 842/1693] 10.222.0.27 () {54 vars in 961 bytes} [Fri Aug 12 12:19:02 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:19:02.619618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa2481-6a0f-44a0-bba6-2aa65b8d2fd7 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:02.620530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa2481-6a0f-44a0-bba6-2aa65b8d2fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:02.620530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa2481-6a0f-44a0-bba6-2aa65b8d2fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:02.679378 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-74aa2481-6a0f-44a0-bba6-2aa65b8d2fd7 None None] MFA Rules not processed for user `5ee8ba21bc184feeaafacab3e69e87bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:02.682146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-74aa2481-6a0f-44a0-bba6-2aa65b8d2fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:02.686944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de627f9-5d16-434d-83eb-98aedb212980 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:02.689061 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4de627f9-5d16-434d-83eb-98aedb212980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:02.735952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de627f9-5d16-434d-83eb-98aedb212980 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:02.820823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f09b6-7691-4cdb-81c3-8d034e18d0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:02.821598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f09b6-7691-4cdb-81c3-8d034e18d0ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:02.821979 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f09b6-7691-4cdb-81c3-8d034e18d0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:02.823573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f09b6-7691-4cdb-81c3-8d034e18d0ff admin admin] PATH_INFO: `/v3/projects/f95d088636e74674b2c3e5bd7e04129b` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:02.828933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f09b6-7691-4cdb-81c3-8d034e18d0ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f95d088636e74674b2c3e5bd7e04129b)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:02.850169 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f09b6-7691-4cdb-81c3-8d034e18d0ff admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:02.928251 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 852/1694] 10.222.0.27 () {58 vars in 1058 bytes} [Fri Aug 12 12:19:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:02.945757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa2481-6a0f-44a0-bba6-2aa65b8d2fd7 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:02.990062 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa2481-6a0f-44a0-bba6-2aa65b8d2fd7 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:03.011934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f099f9a-2c88-4c72-a05f-a499ff7e8e07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ee8ba21bc184feeaafacab3e69e87bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:03.012738 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f099f9a-2c88-4c72-a05f-a499ff7e8e07 service placement] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:03.017037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f099f9a-2c88-4c72-a05f-a499ff7e8e07 service placement] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:03.019526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f099f9a-2c88-4c72-a05f-a499ff7e8e07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:03.027586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0f099f9a-2c88-4c72-a05f-a499ff7e8e07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:03.070552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f099f9a-2c88-4c72-a05f-a499ff7e8e07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:03.076576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f099f9a-2c88-4c72-a05f-a499ff7e8e07 service placement] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:03.116262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 853/1695] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:03.139649 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f32f09b6-7691-4cdb-81c3-8d034e18d0ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f95d088636e74674b2c3e5bd7e04129b', 'request_id': 'req-f32f09b6-7691-4cdb-81c3-8d034e18d0ff'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:03.183880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f32f09b6-7691-4cdb-81c3-8d034e18d0ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f95d088636e74674b2c3e5bd7e04129b', 'request_id': 'req-f32f09b6-7691-4cdb-81c3-8d034e18d0ff'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:03.183880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:03.214111 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f32f09b6-7691-4cdb-81c3-8d034e18d0ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f95d088636e74674b2c3e5bd7e04129b', 'request_id': 'req-f32f09b6-7691-4cdb-81c3-8d034e18d0ff'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:03.303070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f32f09b6-7691-4cdb-81c3-8d034e18d0ff admin admin] The token cache is being invalidate because project f95d088636e74674b2c3e5bd7e04129b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:03.303583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f32f09b6-7691-4cdb-81c3-8d034e18d0ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:03.304495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f32f09b6-7691-4cdb-81c3-8d034e18d0ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:03.309316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 843/1696] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:02 2022] DELETE /identity/v3/projects/f95d088636e74674b2c3e5bd7e04129b => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:03.310556 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f099f9a-2c88-4c72-a05f-a499ff7e8e07 service placement] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:03.333468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f099f9a-2c88-4c72-a05f-a499ff7e8e07 service placement] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:03.341857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b24d6-ab1e-407c-bbe2-a55c202ab481 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:03.342201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b24d6-ab1e-407c-bbe2-a55c202ab481 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:03.342421 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b24d6-ab1e-407c-bbe2-a55c202ab481 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:03.344954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 844/1697] 10.222.0.27 () {54 vars in 912 bytes} [Fri Aug 12 12:19:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:19:03.353627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea334db-2d59-4a26-8d34-9c84821157c1 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:03.354089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea334db-2d59-4a26-8d34-9c84821157c1 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:03.355244 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea334db-2d59-4a26-8d34-9c84821157c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:03.419683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:03.425211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-bea334db-2d59-4a26-8d34-9c84821157c1 None None] MFA Rules not processed for user `745fad9c109a4280906eb0349849117d`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:03.447211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-bea334db-2d59-4a26-8d34-9c84821157c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:03.565963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f37f2-88a4-4828-bbb0-51475a15cf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:03.568407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f37f2-88a4-4828-bbb0-51475a15cf26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:03.570940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f37f2-88a4-4828-bbb0-51475a15cf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:03.573074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f37f2-88a4-4828-bbb0-51475a15cf26 admin admin] PATH_INFO: `/v3/users/ee95d39e96284d5f9a99c5a45fd0259d` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:03.576415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f37f2-88a4-4828-bbb0-51475a15cf26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee95d39e96284d5f9a99c5a45fd0259d)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:03.584015 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f37f2-88a4-4828-bbb0-51475a15cf26 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:03.635060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 845/1698] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Aug 12 12:19:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:03.649968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea334db-2d59-4a26-8d34-9c84821157c1 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:03.653284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-bea334db-2d59-4a26-8d34-9c84821157c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:03.670628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea334db-2d59-4a26-8d34-9c84821157c1 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:03.688085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cd031-720d-454a-8472-12dcc4182d51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:03.698192 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cd031-720d-454a-8472-12dcc4182d51 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:03.698494 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cd031-720d-454a-8472-12dcc4182d51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:03.704198 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cd031-720d-454a-8472-12dcc4182d51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:03.734590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9cd031-720d-454a-8472-12dcc4182d51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:03.736442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9cd031-720d-454a-8472-12dcc4182d51 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:03.749138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 846/1699] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:03.829060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7b7f37f2-88a4-4828-bbb0-51475a15cf26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee95d39e96284d5f9a99c5a45fd0259d'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:03.921423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7b7f37f2-88a4-4828-bbb0-51475a15cf26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee95d39e96284d5f9a99c5a45fd0259d'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:03.923103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7b7f37f2-88a4-4828-bbb0-51475a15cf26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee95d39e96284d5f9a99c5a45fd0259d'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:03.957531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7b7f37f2-88a4-4828-bbb0-51475a15cf26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee95d39e96284d5f9a99c5a45fd0259d'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:04.045247 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7b7f37f2-88a4-4828-bbb0-51475a15cf26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee95d39e96284d5f9a99c5a45fd0259d'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:04.091728 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 854/1700] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:03 2022] DELETE /identity/v3/users/ee95d39e96284d5f9a99c5a45fd0259d => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:04.595080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe5779-70eb-405a-b4ba-cd2862b0264e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:04.595080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe5779-70eb-405a-b4ba-cd2862b0264e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:04.595080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe5779-70eb-405a-b4ba-cd2862b0264e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:04.650242 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-43fe5779-70eb-405a-b4ba-cd2862b0264e None None] MFA Rules not processed for user `0156c689e109489d9265dd3dd4d5cfde`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:04.652386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-43fe5779-70eb-405a-b4ba-cd2862b0264e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:04.848766 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 847/1701] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:04.879206 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f37f2-88a4-4828-bbb0-51475a15cf26 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:04.881197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7b7f37f2-88a4-4828-bbb0-51475a15cf26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:05.157128 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f37f2-88a4-4828-bbb0-51475a15cf26 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:05.164400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38fdda-5e66-4d0e-aebf-020a8fe8046f None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:05.164769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38fdda-5e66-4d0e-aebf-020a8fe8046f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:05.165053 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38fdda-5e66-4d0e-aebf-020a8fe8046f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:05.173395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7872cc0-0777-43c4-9a6a-24a853a9c034 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:05.174073 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7872cc0-0777-43c4-9a6a-24a853a9c034 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:05.174994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7872cc0-0777-43c4-9a6a-24a853a9c034 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:05.175277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7872cc0-0777-43c4-9a6a-24a853a9c034 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:05.196461 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7872cc0-0777-43c4-9a6a-24a853a9c034 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:05.200146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7872cc0-0777-43c4-9a6a-24a853a9c034 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:05.233403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 855/1702] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:05.273933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-4f38fdda-5e66-4d0e-aebf-020a8fe8046f None None] MFA Rules not processed for user `383cbf83186747a5b81a12ed5288c39a`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:05.274626 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4f38fdda-5e66-4d0e-aebf-020a8fe8046f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:05.312854 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e572a52-88d1-4ab6-802e-c65d7499a248 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:05.313460 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e572a52-88d1-4ab6-802e-c65d7499a248 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:05.314044 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e572a52-88d1-4ab6-802e-c65d7499a248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:05.371661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-0e572a52-88d1-4ab6-802e-c65d7499a248 None None] MFA Rules not processed for user `099b87a6c6b8409ab188c0b71ded7f55`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:05.379520 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0e572a52-88d1-4ab6-802e-c65d7499a248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:05.545096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 848/1703] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:05.557958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:05.646495 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 856/1704] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:05.691955 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38fdda-5e66-4d0e-aebf-020a8fe8046f None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:05.694020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4f38fdda-5e66-4d0e-aebf-020a8fe8046f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:05.699393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:05.781495 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e572a52-88d1-4ab6-802e-c65d7499a248 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:05.783166 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0e572a52-88d1-4ab6-802e-c65d7499a248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:05.808958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e572a52-88d1-4ab6-802e-c65d7499a248 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:05.825679 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22a427-74c3-4b05-b3fd-65439aac2238 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:05.827046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22a427-74c3-4b05-b3fd-65439aac2238 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:05.827606 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22a427-74c3-4b05-b3fd-65439aac2238 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:05.827953 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22a427-74c3-4b05-b3fd-65439aac2238 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:05.842382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22a427-74c3-4b05-b3fd-65439aac2238 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:05.844660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22a427-74c3-4b05-b3fd-65439aac2238 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:05.888404 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 857/1705] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:05 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:19:05.917304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38fdda-5e66-4d0e-aebf-020a8fe8046f None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:05.924147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d92d1-31ba-4ce1-97ad-362be28ef8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:05.925169 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d92d1-31ba-4ce1-97ad-362be28ef8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:05.925865 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d92d1-31ba-4ce1-97ad-362be28ef8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:05.926423 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d92d1-31ba-4ce1-97ad-362be28ef8d8 admin admin] PATH_INFO: `/v3/projects/ba0ab342961d4cb8a7bf2bf76ad7eb2e` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:05.929593 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d92d1-31ba-4ce1-97ad-362be28ef8d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba0ab342961d4cb8a7bf2bf76ad7eb2e)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:05.931976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d92d1-31ba-4ce1-97ad-362be28ef8d8 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:06.127858 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e25d92d1-31ba-4ce1-97ad-362be28ef8d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba0ab342961d4cb8a7bf2bf76ad7eb2e', 'request_id': 'req-e25d92d1-31ba-4ce1-97ad-362be28ef8d8'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:06.155693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e25d92d1-31ba-4ce1-97ad-362be28ef8d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba0ab342961d4cb8a7bf2bf76ad7eb2e', 'request_id': 'req-e25d92d1-31ba-4ce1-97ad-362be28ef8d8'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:06.172987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e25d92d1-31ba-4ce1-97ad-362be28ef8d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba0ab342961d4cb8a7bf2bf76ad7eb2e', 'request_id': 'req-e25d92d1-31ba-4ce1-97ad-362be28ef8d8'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:06.293539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e25d92d1-31ba-4ce1-97ad-362be28ef8d8 admin admin] The token cache is being invalidate because project ba0ab342961d4cb8a7bf2bf76ad7eb2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:06.293539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e25d92d1-31ba-4ce1-97ad-362be28ef8d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:06.293539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e25d92d1-31ba-4ce1-97ad-362be28ef8d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:06.301850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 849/1706] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:05 2022] DELETE /identity/v3/projects/ba0ab342961d4cb8a7bf2bf76ad7eb2e => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:06.303248 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b54bc3-344f-4e01-b543-3c341553a5c3 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:06.304130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b54bc3-344f-4e01-b543-3c341553a5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:06.304558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b54bc3-344f-4e01-b543-3c341553a5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:06.433248 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-a0b54bc3-344f-4e01-b543-3c341553a5c3 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:06.441860 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a0b54bc3-344f-4e01-b543-3c341553a5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:06.675959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 858/1707] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:06.683762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6940b-13af-4823-8a1c-0efec87bdbaf None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:06.684249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6940b-13af-4823-8a1c-0efec87bdbaf None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:06.685717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6940b-13af-4823-8a1c-0efec87bdbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:06.739224 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-edd6940b-13af-4823-8a1c-0efec87bdbaf None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:06.743815 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-edd6940b-13af-4823-8a1c-0efec87bdbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:06.763421 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 850/1708] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:06.768041 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b54bc3-344f-4e01-b543-3c341553a5c3 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:06.782138 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b54bc3-344f-4e01-b543-3c341553a5c3 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:06.794861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4b762-6559-47e9-86d0-6b4c2a62c1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:06.795558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4b762-6559-47e9-86d0-6b4c2a62c1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:06.804085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4b762-6559-47e9-86d0-6b4c2a62c1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:06.808814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4b762-6559-47e9-86d0-6b4c2a62c1c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:06.808814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4b762-6559-47e9-86d0-6b4c2a62c1c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:19:06.808814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4b762-6559-47e9-86d0-6b4c2a62c1c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:06.808814 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4b762-6559-47e9-86d0-6b4c2a62c1c6 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:06.838506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 859/1709] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:19:06.843308 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6940b-13af-4823-8a1c-0efec87bdbaf None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:06.858877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6940b-13af-4823-8a1c-0efec87bdbaf None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:06.870901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b430291-a4e1-4e8d-8c62-cc5d1a049062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:06.874011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b430291-a4e1-4e8d-8c62-cc5d1a049062 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:06.874453 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b430291-a4e1-4e8d-8c62-cc5d1a049062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:06.874895 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b430291-a4e1-4e8d-8c62-cc5d1a049062 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:06.877783 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b430291-a4e1-4e8d-8c62-cc5d1a049062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:06.880111 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b430291-a4e1-4e8d-8c62-cc5d1a049062 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:07.081769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 851/1710] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:06 2022] POST /identity/v3/projects => generated 387 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:07.086146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4b762-6559-47e9-86d0-6b4c2a62c1c6 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:07.102148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4b762-6559-47e9-86d0-6b4c2a62c1c6 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:07.240781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492945d-1646-4d7c-b2b1-666591be06a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:07.241844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492945d-1646-4d7c-b2b1-666591be06a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:07.242436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492945d-1646-4d7c-b2b1-666591be06a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:07.242883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492945d-1646-4d7c-b2b1-666591be06a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:07.244143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492945d-1646-4d7c-b2b1-666591be06a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:07.250368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492945d-1646-4d7c-b2b1-666591be06a8 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:07.401969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 860/1711] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:07 2022] POST /identity/v3/users => generated 294 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:07.406203 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b430291-a4e1-4e8d-8c62-cc5d1a049062 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:07.422564 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b430291-a4e1-4e8d-8c62-cc5d1a049062 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:07.434379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ebb08-ef2d-4971-97ef-e6cdeab758b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:07.437003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ebb08-ef2d-4971-97ef-e6cdeab758b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:07.437003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ebb08-ef2d-4971-97ef-e6cdeab758b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:07.437003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ebb08-ef2d-4971-97ef-e6cdeab758b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:07.437360 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ebb08-ef2d-4971-97ef-e6cdeab758b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:07.438994 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ebb08-ef2d-4971-97ef-e6cdeab758b4 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:07.457748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492945d-1646-4d7c-b2b1-666591be06a8 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:07.458129 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 852/1712] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:07.469589 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492945d-1646-4d7c-b2b1-666591be06a8 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:07.483996 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbefac8-2b27-4641-87f4-3b335954a4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:07.485962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbefac8-2b27-4641-87f4-3b335954a4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:07.486370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbefac8-2b27-4641-87f4-3b335954a4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:07.486744 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbefac8-2b27-4641-87f4-3b335954a4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:07.487771 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbefac8-2b27-4641-87f4-3b335954a4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:07.490027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbefac8-2b27-4641-87f4-3b335954a4c5 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:07.518268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 861/1713] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:07.524293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ebb08-ef2d-4971-97ef-e6cdeab758b4 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:07.542632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ebb08-ef2d-4971-97ef-e6cdeab758b4 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:07.558876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4918d-1f19-471f-8809-72b3fa9bcf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:07.561488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4918d-1f19-471f-8809-72b3fa9bcf52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:07.561488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4918d-1f19-471f-8809-72b3fa9bcf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:07.561488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4918d-1f19-471f-8809-72b3fa9bcf52 admin admin] PATH_INFO: `/v3/projects/97564533f3fd4a7ab2b8e9ce33aa4acc/users/cbf106687a7c4801b356ab997c08b896/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:07.594797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4918d-1f19-471f-8809-72b3fa9bcf52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97564533f3fd4a7ab2b8e9ce33aa4acc, user_id=cbf106687a7c4801b356ab997c08b896, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:07.599560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4918d-1f19-471f-8809-72b3fa9bcf52 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:07.753392 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 853/1714] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:07 2022] PUT /identity/v3/projects/97564533f3fd4a7ab2b8e9ce33aa4acc/users/cbf106687a7c4801b356ab997c08b896/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:07.766757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa0180-dab4-4e6a-810e-5dd755f16c46 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:07.767203 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa0180-dab4-4e6a-810e-5dd755f16c46 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:07.768228 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa0180-dab4-4e6a-810e-5dd755f16c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:07.846423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-2faa0180-dab4-4e6a-810e-5dd755f16c46 None None] MFA Rules not processed for user `cbf106687a7c4801b356ab997c08b896`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:07.848118 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2faa0180-dab4-4e6a-810e-5dd755f16c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:07.941259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:08.032072 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 862/1715] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:08.037103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4918d-1f19-471f-8809-72b3fa9bcf52 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:08.141081 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:08.194831 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4918d-1f19-471f-8809-72b3fa9bcf52 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:08.200181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feddcac-b893-4c91-a153-38c298d0321e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:08.200964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feddcac-b893-4c91-a153-38c298d0321e admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:08.202230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feddcac-b893-4c91-a153-38c298d0321e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:08.202631 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feddcac-b893-4c91-a153-38c298d0321e admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:08.203360 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feddcac-b893-4c91-a153-38c298d0321e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:08.205627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feddcac-b893-4c91-a153-38c298d0321e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:08.372223 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa0180-dab4-4e6a-810e-5dd755f16c46 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:08.373676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 854/1716] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:08 2022] POST /identity/v3/projects => generated 385 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:08.390700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa0180-dab4-4e6a-810e-5dd755f16c46 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:08.537476 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10841851-4695-495f-bd42-b7fa1cf56a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:08.538112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10841851-4695-495f-bd42-b7fa1cf56a43 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:08.538452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10841851-4695-495f-bd42-b7fa1cf56a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:08.538644 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10841851-4695-495f-bd42-b7fa1cf56a43 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:08.539287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10841851-4695-495f-bd42-b7fa1cf56a43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:08.541553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10841851-4695-495f-bd42-b7fa1cf56a43 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:08.686526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 863/1717] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:08 2022] POST /identity/v3/users => generated 299 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:08.688386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feddcac-b893-4c91-a153-38c298d0321e admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:08.703500 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feddcac-b893-4c91-a153-38c298d0321e admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:08.720135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482866d2-3eba-4e63-823e-a8a2c87636fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:08.720907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482866d2-3eba-4e63-823e-a8a2c87636fa admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:08.721197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482866d2-3eba-4e63-823e-a8a2c87636fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:08.721467 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482866d2-3eba-4e63-823e-a8a2c87636fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:08.722116 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482866d2-3eba-4e63-823e-a8a2c87636fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:08.724203 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482866d2-3eba-4e63-823e-a8a2c87636fa admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:08.757473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10841851-4695-495f-bd42-b7fa1cf56a43 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:08.773156 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 855/1718] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:08.776152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10841851-4695-495f-bd42-b7fa1cf56a43 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:08.799825 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba503d58-c8fe-4a72-b727-9f63243b9f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:08.800738 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba503d58-c8fe-4a72-b727-9f63243b9f21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:08.801019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba503d58-c8fe-4a72-b727-9f63243b9f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:08.801668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba503d58-c8fe-4a72-b727-9f63243b9f21 admin admin] PATH_INFO: `/v3/projects/df51ef98a1934686998e1ad5467894c4/users/64259aa7c6a9441b957135c475f2ff32/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:08.821647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba503d58-c8fe-4a72-b727-9f63243b9f21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df51ef98a1934686998e1ad5467894c4, user_id=64259aa7c6a9441b957135c475f2ff32, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:08.824817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba503d58-c8fe-4a72-b727-9f63243b9f21 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:08.921981 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482866d2-3eba-4e63-823e-a8a2c87636fa admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:08.922439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 864/1719] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:08 2022] PUT /identity/v3/projects/df51ef98a1934686998e1ad5467894c4/users/64259aa7c6a9441b957135c475f2ff32/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:08.933540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482866d2-3eba-4e63-823e-a8a2c87636fa admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:09.001767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dbbb3-be83-484c-8fae-389e71c144bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:09.002389 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dbbb3-be83-484c-8fae-389e71c144bf admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:09.002674 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dbbb3-be83-484c-8fae-389e71c144bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:09.002943 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dbbb3-be83-484c-8fae-389e71c144bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:09.003594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8dbbb3-be83-484c-8fae-389e71c144bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:09.005444 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8dbbb3-be83-484c-8fae-389e71c144bf admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:09.016780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 856/1720] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:09.021370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba503d58-c8fe-4a72-b727-9f63243b9f21 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:09.032030 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba503d58-c8fe-4a72-b727-9f63243b9f21 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:09.037780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc190e-3bc5-48bf-812c-1cabc35623ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:09.038944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc190e-3bc5-48bf-812c-1cabc35623ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:09.039415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc190e-3bc5-48bf-812c-1cabc35623ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:09.039945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc190e-3bc5-48bf-812c-1cabc35623ff admin admin] PATH_INFO: `/v3/projects/df51ef98a1934686998e1ad5467894c4/users/64259aa7c6a9441b957135c475f2ff32/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:09.042404 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc190e-3bc5-48bf-812c-1cabc35623ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=df51ef98a1934686998e1ad5467894c4, user_id=64259aa7c6a9441b957135c475f2ff32, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:09.045170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc190e-3bc5-48bf-812c-1cabc35623ff admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:09.056123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 865/1721] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:09 2022] PUT /identity/v3/projects/df51ef98a1934686998e1ad5467894c4/users/64259aa7c6a9441b957135c475f2ff32/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:09.067303 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bee9d-a82c-4cca-ad0b-1f2b05bd8977 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:09.067841 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bee9d-a82c-4cca-ad0b-1f2b05bd8977 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:09.068447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bee9d-a82c-4cca-ad0b-1f2b05bd8977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:09.102817 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-283bee9d-a82c-4cca-ad0b-1f2b05bd8977 None None] MFA Rules not processed for user `64259aa7c6a9441b957135c475f2ff32`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:09.104230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-283bee9d-a82c-4cca-ad0b-1f2b05bd8977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:09.300230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 857/1722] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:09.311352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc190e-3bc5-48bf-812c-1cabc35623ff admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:09.319590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-49cc190e-3bc5-48bf-812c-1cabc35623ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:09.442761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc190e-3bc5-48bf-812c-1cabc35623ff admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:09.446773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85764c95-a9fa-42c1-a33c-09ac330aadf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:09.447521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85764c95-a9fa-42c1-a33c-09ac330aadf3 service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:09.448019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85764c95-a9fa-42c1-a33c-09ac330aadf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:09.448451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85764c95-a9fa-42c1-a33c-09ac330aadf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:09.462598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85764c95-a9fa-42c1-a33c-09ac330aadf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:09.468503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85764c95-a9fa-42c1-a33c-09ac330aadf3 service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:09.484411 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 866/1723] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:09.594434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bee9d-a82c-4cca-ad0b-1f2b05bd8977 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:09.708427 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bee9d-a82c-4cca-ad0b-1f2b05bd8977 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:09.710312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213ed0b-94b8-49cf-9f28-2d75990e6566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:09.711576 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213ed0b-94b8-49cf-9f28-2d75990e6566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:09.711860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213ed0b-94b8-49cf-9f28-2d75990e6566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:09.712185 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213ed0b-94b8-49cf-9f28-2d75990e6566 admin admin] PATH_INFO: `/v3/users/cbf106687a7c4801b356ab997c08b896` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:09.713768 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213ed0b-94b8-49cf-9f28-2d75990e6566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbf106687a7c4801b356ab997c08b896)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:09.716305 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213ed0b-94b8-49cf-9f28-2d75990e6566 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:09.946691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4213ed0b-94b8-49cf-9f28-2d75990e6566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbf106687a7c4801b356ab997c08b896'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:09.950364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:09.951877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4213ed0b-94b8-49cf-9f28-2d75990e6566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbf106687a7c4801b356ab997c08b896'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:09.998843 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663be0aa-d427-4f8e-ba28-d31c3b2d7e80 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:09.999195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663be0aa-d427-4f8e-ba28-d31c3b2d7e80 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:09.999481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663be0aa-d427-4f8e-ba28-d31c3b2d7e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:10.028936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-663be0aa-d427-4f8e-ba28-d31c3b2d7e80 None None] MFA Rules not processed for user `0156c689e109489d9265dd3dd4d5cfde`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:10.032373 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-663be0aa-d427-4f8e-ba28-d31c3b2d7e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:10.061816 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4213ed0b-94b8-49cf-9f28-2d75990e6566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbf106687a7c4801b356ab997c08b896'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:10.069178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:10.078415 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4213ed0b-94b8-49cf-9f28-2d75990e6566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbf106687a7c4801b356ab997c08b896'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:10.110609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4213ed0b-94b8-49cf-9f28-2d75990e6566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbf106687a7c4801b356ab997c08b896'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:10.123529 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 858/1724] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:09 2022] DELETE /identity/v3/users/cbf106687a7c4801b356ab997c08b896 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:10.130842 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f1bc2-f33e-4efe-921c-f50887979b81 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:10.131343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f1bc2-f33e-4efe-921c-f50887979b81 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:10.131793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f1bc2-f33e-4efe-921c-f50887979b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:10.161326 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-8e4f1bc2-f33e-4efe-921c-f50887979b81 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:10.162671 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 867/1725] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:10.166164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8e4f1bc2-f33e-4efe-921c-f50887979b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:10.306320 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 859/1726] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:10.358831 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663be0aa-d427-4f8e-ba28-d31c3b2d7e80 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:10.360081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-663be0aa-d427-4f8e-ba28-d31c3b2d7e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:10.486567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663be0aa-d427-4f8e-ba28-d31c3b2d7e80 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:10.553104 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279947a7-eb9e-4c09-8eda-7fb7d169c0d0 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:10.553508 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279947a7-eb9e-4c09-8eda-7fb7d169c0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:10.553928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279947a7-eb9e-4c09-8eda-7fb7d169c0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:10.608735 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-279947a7-eb9e-4c09-8eda-7fb7d169c0d0 None None] MFA Rules not processed for user `383cbf83186747a5b81a12ed5288c39a`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:10.620159 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-279947a7-eb9e-4c09-8eda-7fb7d169c0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:10.643314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c843e2f-de4c-4e22-9893-57faf617a72b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:10.644050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c843e2f-de4c-4e22-9893-57faf617a72b service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:10.644323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c843e2f-de4c-4e22-9893-57faf617a72b service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:10.645233 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c843e2f-de4c-4e22-9893-57faf617a72b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:10.661632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c843e2f-de4c-4e22-9893-57faf617a72b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:10.665408 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c843e2f-de4c-4e22-9893-57faf617a72b service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:10.688341 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 868/1727] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:10.728120 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 860/1728] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:11.401253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:11.908072 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c843e2f-de4c-4e22-9893-57faf617a72b service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:11.913754 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2c843e2f-de4c-4e22-9893-57faf617a72b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:11.931731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c843e2f-de4c-4e22-9893-57faf617a72b service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:11.937795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d882e7e-06cf-407c-9564-1b5aa3aa9585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:11.938847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d882e7e-06cf-407c-9564-1b5aa3aa9585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:11.939804 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d882e7e-06cf-407c-9564-1b5aa3aa9585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:11.941049 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d882e7e-06cf-407c-9564-1b5aa3aa9585 admin admin] PATH_INFO: `/v3/projects/97564533f3fd4a7ab2b8e9ce33aa4acc` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:11.944404 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d882e7e-06cf-407c-9564-1b5aa3aa9585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97564533f3fd4a7ab2b8e9ce33aa4acc)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:11.946703 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d882e7e-06cf-407c-9564-1b5aa3aa9585 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:12.146105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7d882e7e-06cf-407c-9564-1b5aa3aa9585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97564533f3fd4a7ab2b8e9ce33aa4acc', 'request_id': 'req-7d882e7e-06cf-407c-9564-1b5aa3aa9585'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:12.162872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7d882e7e-06cf-407c-9564-1b5aa3aa9585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97564533f3fd4a7ab2b8e9ce33aa4acc', 'request_id': 'req-7d882e7e-06cf-407c-9564-1b5aa3aa9585'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:12.174376 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7d882e7e-06cf-407c-9564-1b5aa3aa9585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97564533f3fd4a7ab2b8e9ce33aa4acc', 'request_id': 'req-7d882e7e-06cf-407c-9564-1b5aa3aa9585'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:12.401092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7d882e7e-06cf-407c-9564-1b5aa3aa9585 admin admin] The token cache is being invalidate because project 97564533f3fd4a7ab2b8e9ce33aa4acc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:12.401552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7d882e7e-06cf-407c-9564-1b5aa3aa9585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:12.402527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7d882e7e-06cf-407c-9564-1b5aa3aa9585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:12.409735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 869/1729] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:11 2022] DELETE /identity/v3/projects/97564533f3fd4a7ab2b8e9ce33aa4acc => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:12.415088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279947a7-eb9e-4c09-8eda-7fb7d169c0d0 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:12.438131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279947a7-eb9e-4c09-8eda-7fb7d169c0d0 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:12.563378 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abddeaa-cae9-45d0-b10b-e8bfaf397088 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:12.563667 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abddeaa-cae9-45d0-b10b-e8bfaf397088 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:12.564710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abddeaa-cae9-45d0-b10b-e8bfaf397088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:12.581032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:12.618042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0684d-8a48-4f4a-a255-8e731f7ceb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:12.619521 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0684d-8a48-4f4a-a255-8e731f7ceb06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:12.622397 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0684d-8a48-4f4a-a255-8e731f7ceb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:12.622397 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0684d-8a48-4f4a-a255-8e731f7ceb06 admin admin] PATH_INFO: `/v3/users/64259aa7c6a9441b957135c475f2ff32` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:12.622397 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0684d-8a48-4f4a-a255-8e731f7ceb06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64259aa7c6a9441b957135c475f2ff32)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:12.623951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0684d-8a48-4f4a-a255-8e731f7ceb06 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:12.785392 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-4abddeaa-cae9-45d0-b10b-e8bfaf397088 None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:12.787408 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4abddeaa-cae9-45d0-b10b-e8bfaf397088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:12.931857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e7e0684d-8a48-4f4a-a255-8e731f7ceb06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64259aa7c6a9441b957135c475f2ff32'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:12.933337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e7e0684d-8a48-4f4a-a255-8e731f7ceb06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64259aa7c6a9441b957135c475f2ff32'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:13.036165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 870/1730] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Aug 12 12:19:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:13.037139 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abddeaa-cae9-45d0-b10b-e8bfaf397088 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:13.038594 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4abddeaa-cae9-45d0-b10b-e8bfaf397088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:13.061777 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abddeaa-cae9-45d0-b10b-e8bfaf397088 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:13.068384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab5a7f-c605-4e36-843d-9af7e79309fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:13.069413 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab5a7f-c605-4e36-843d-9af7e79309fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:13.069889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab5a7f-c605-4e36-843d-9af7e79309fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:13.070350 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab5a7f-c605-4e36-843d-9af7e79309fd admin admin] PATH_INFO: `/v3/users/aab3f62ff9bc461fa47a34d851a86642` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:13.071954 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab5a7f-c605-4e36-843d-9af7e79309fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab3f62ff9bc461fa47a34d851a86642)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:13.074408 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab5a7f-c605-4e36-843d-9af7e79309fd admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:13.093400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e7e0684d-8a48-4f4a-a255-8e731f7ceb06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64259aa7c6a9441b957135c475f2ff32'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:13.133360 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e7e0684d-8a48-4f4a-a255-8e731f7ceb06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64259aa7c6a9441b957135c475f2ff32'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:13.156714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e7e0684d-8a48-4f4a-a255-8e731f7ceb06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64259aa7c6a9441b957135c475f2ff32'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:13.184032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 861/1731] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:12 2022] DELETE /identity/v3/users/64259aa7c6a9441b957135c475f2ff32 => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:13.388577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-feab5a7f-c605-4e36-843d-9af7e79309fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab3f62ff9bc461fa47a34d851a86642'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:13.473090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-feab5a7f-c605-4e36-843d-9af7e79309fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab3f62ff9bc461fa47a34d851a86642'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:13.475852 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-feab5a7f-c605-4e36-843d-9af7e79309fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab3f62ff9bc461fa47a34d851a86642'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:13.485477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0684d-8a48-4f4a-a255-8e731f7ceb06 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:13.487123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-e7e0684d-8a48-4f4a-a255-8e731f7ceb06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:13.491310 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-feab5a7f-c605-4e36-843d-9af7e79309fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab3f62ff9bc461fa47a34d851a86642'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:13.511494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-feab5a7f-c605-4e36-843d-9af7e79309fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab3f62ff9bc461fa47a34d851a86642'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:13.528591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 871/1732] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:13 2022] DELETE /identity/v3/users/aab3f62ff9bc461fa47a34d851a86642 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:13.533424 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fecc5-1841-4a5d-b388-d0a62ce4135e None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:13.535091 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fecc5-1841-4a5d-b388-d0a62ce4135e None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:13.535694 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fecc5-1841-4a5d-b388-d0a62ce4135e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:13.566859 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-439fecc5-1841-4a5d-b388-d0a62ce4135e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:13.568932 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-439fecc5-1841-4a5d-b388-d0a62ce4135e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:13.676793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 872/1733] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:13.681009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0684d-8a48-4f4a-a255-8e731f7ceb06 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:13.685997 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fecc5-1841-4a5d-b388-d0a62ce4135e None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:13.688090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-439fecc5-1841-4a5d-b388-d0a62ce4135e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:13.766034 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fecc5-1841-4a5d-b388-d0a62ce4135e None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:13.773293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55799c80-389f-4298-862d-59a7c310f3a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:13.774445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55799c80-389f-4298-862d-59a7c310f3a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:13.774726 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55799c80-389f-4298-862d-59a7c310f3a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:13.775004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55799c80-389f-4298-862d-59a7c310f3a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:13.779895 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7fad5-8997-4e23-937a-22af230a34cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:13.780729 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7fad5-8997-4e23-937a-22af230a34cc service swift] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:13.781019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7fad5-8997-4e23-937a-22af230a34cc service swift] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:13.781299 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7fad5-8997-4e23-937a-22af230a34cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:13.784988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-18c7fad5-8997-4e23-937a-22af230a34cc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:13.786090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55799c80-389f-4298-862d-59a7c310f3a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:13.788219 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55799c80-389f-4298-862d-59a7c310f3a6 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:13.817869 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 873/1734] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:19:13.884798 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7fad5-8997-4e23-937a-22af230a34cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:13.886995 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7fad5-8997-4e23-937a-22af230a34cc service swift] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:13.897580 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 862/1735] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:19:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 12 12:19:14.042482 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55799c80-389f-4298-862d-59a7c310f3a6 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:14.044132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-55799c80-389f-4298-862d-59a7c310f3a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:14.066077 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55799c80-389f-4298-862d-59a7c310f3a6 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:14.073893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255f00e-a26e-4d01-8ba4-270c1602afaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:14.075379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255f00e-a26e-4d01-8ba4-270c1602afaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:14.076000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255f00e-a26e-4d01-8ba4-270c1602afaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:14.076550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255f00e-a26e-4d01-8ba4-270c1602afaf admin admin] PATH_INFO: `/v3/users/f62af74954884d0693a4877a869f02fb` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:14.082430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255f00e-a26e-4d01-8ba4-270c1602afaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f62af74954884d0693a4877a869f02fb)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:14.085144 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255f00e-a26e-4d01-8ba4-270c1602afaf admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:14.328821 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:14.385413 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d255f00e-a26e-4d01-8ba4-270c1602afaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f62af74954884d0693a4877a869f02fb'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:14.490977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f94b6-cb64-4ab5-9d99-c5153369ebf9 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:14.491972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f94b6-cb64-4ab5-9d99-c5153369ebf9 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:14.492546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f94b6-cb64-4ab5-9d99-c5153369ebf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:14.518628 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d255f00e-a26e-4d01-8ba4-270c1602afaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f62af74954884d0693a4877a869f02fb'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:14.519250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d255f00e-a26e-4d01-8ba4-270c1602afaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f62af74954884d0693a4877a869f02fb'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:14.532201 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d255f00e-a26e-4d01-8ba4-270c1602afaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f62af74954884d0693a4877a869f02fb'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:14.565456 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d255f00e-a26e-4d01-8ba4-270c1602afaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f62af74954884d0693a4877a869f02fb'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:14.569583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-423f94b6-cb64-4ab5-9d99-c5153369ebf9 None None] MFA Rules not processed for user `745fad9c109a4280906eb0349849117d`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:14.582668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-423f94b6-cb64-4ab5-9d99-c5153369ebf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:14.599893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 874/1736] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:14 2022] DELETE /identity/v3/users/f62af74954884d0693a4877a869f02fb => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:14.599893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ba0f6-0a4f-4f33-a38a-276afdef5f3d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:14.599893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ba0f6-0a4f-4f33-a38a-276afdef5f3d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:14.599893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ba0f6-0a4f-4f33-a38a-276afdef5f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:14.610987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 863/1737] 10.222.0.46 () {58 vars in 1015 bytes} [Fri Aug 12 12:19:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:14.628412 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f94b6-cb64-4ab5-9d99-c5153369ebf9 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:14.632216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-423f94b6-cb64-4ab5-9d99-c5153369ebf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:14.675430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-bc9ba0f6-0a4f-4f33-a38a-276afdef5f3d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:14.689559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-bc9ba0f6-0a4f-4f33-a38a-276afdef5f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:14.700252 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f94b6-cb64-4ab5-9d99-c5153369ebf9 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:14.827238 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78485e-8c19-4947-9d00-ed35f290e751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:14.831131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78485e-8c19-4947-9d00-ed35f290e751 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:14.832742 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78485e-8c19-4947-9d00-ed35f290e751 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:14.833251 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78485e-8c19-4947-9d00-ed35f290e751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:14.836703 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 875/1738] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:14.847777 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ba0f6-0a4f-4f33-a38a-276afdef5f3d None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:14.851379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f78485e-8c19-4947-9d00-ed35f290e751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:14.870304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f78485e-8c19-4947-9d00-ed35f290e751 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:14.881773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ba0f6-0a4f-4f33-a38a-276afdef5f3d None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:14.894020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcdaef-e0a8-4a5e-93a4-8282be092f3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:14.897514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcdaef-e0a8-4a5e-93a4-8282be092f3e service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:14.898333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcdaef-e0a8-4a5e-93a4-8282be092f3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:14.899413 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcdaef-e0a8-4a5e-93a4-8282be092f3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:14.913971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 864/1739] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:14.928141 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bcdaef-e0a8-4a5e-93a4-8282be092f3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:14.930740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bcdaef-e0a8-4a5e-93a4-8282be092f3e service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:14.951819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 876/1740] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:15.308651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78485e-8c19-4947-9d00-ed35f290e751 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:15.310301 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8f78485e-8c19-4947-9d00-ed35f290e751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:15.357818 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78485e-8c19-4947-9d00-ed35f290e751 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:15.369312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab702078-23d0-4e04-a755-d36ae4bcffa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:15.371538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab702078-23d0-4e04-a755-d36ae4bcffa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:15.371834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab702078-23d0-4e04-a755-d36ae4bcffa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:15.379977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab702078-23d0-4e04-a755-d36ae4bcffa1 admin admin] PATH_INFO: `/v3/projects/fbc30c5ff3ea4fcbb7b176a430f18cf3` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:15.415226 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab702078-23d0-4e04-a755-d36ae4bcffa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbc30c5ff3ea4fcbb7b176a430f18cf3)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:15.430333 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab702078-23d0-4e04-a755-d36ae4bcffa1 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:15.519120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcdaef-e0a8-4a5e-93a4-8282be092f3e service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:15.521598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-01bcdaef-e0a8-4a5e-93a4-8282be092f3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:15.596350 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcdaef-e0a8-4a5e-93a4-8282be092f3e service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:15.753977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:15.753977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:15.753977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:15.753977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33 admin admin] PATH_INFO: `/v3/projects/df51ef98a1934686998e1ad5467894c4` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:15.755270 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df51ef98a1934686998e1ad5467894c4)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:15.755270 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:16.088502 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ab702078-23d0-4e04-a755-d36ae4bcffa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc30c5ff3ea4fcbb7b176a430f18cf3', 'request_id': 'req-ab702078-23d0-4e04-a755-d36ae4bcffa1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:16.114212 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ab702078-23d0-4e04-a755-d36ae4bcffa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc30c5ff3ea4fcbb7b176a430f18cf3', 'request_id': 'req-ab702078-23d0-4e04-a755-d36ae4bcffa1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:16.140480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ab702078-23d0-4e04-a755-d36ae4bcffa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc30c5ff3ea4fcbb7b176a430f18cf3', 'request_id': 'req-ab702078-23d0-4e04-a755-d36ae4bcffa1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:16.144115 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df51ef98a1934686998e1ad5467894c4', 'request_id': 'req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:16.148350 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:16.156577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df51ef98a1934686998e1ad5467894c4', 'request_id': 'req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:16.219246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df51ef98a1934686998e1ad5467894c4', 'request_id': 'req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:16.305714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ab702078-23d0-4e04-a755-d36ae4bcffa1 admin admin] The token cache is being invalidate because project fbc30c5ff3ea4fcbb7b176a430f18cf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:16.305714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ab702078-23d0-4e04-a755-d36ae4bcffa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:16.305714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ab702078-23d0-4e04-a755-d36ae4bcffa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:16.305714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 865/1741] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:15 2022] DELETE /identity/v3/projects/fbc30c5ff3ea4fcbb7b176a430f18cf3 => generated 0 bytes in 993 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:16.305714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab702078-23d0-4e04-a755-d36ae4bcffa1 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:16.306636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-ab702078-23d0-4e04-a755-d36ae4bcffa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:16.435906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33 admin admin] The token cache is being invalidate because project df51ef98a1934686998e1ad5467894c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:16.436438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:16.437615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:16.441354 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 877/1742] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:15 2022] DELETE /identity/v3/projects/df51ef98a1934686998e1ad5467894c4 => generated 0 bytes in 923 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:19:16.444380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:16.450037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:16.512275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab702078-23d0-4e04-a755-d36ae4bcffa1 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:16.595916 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d565e8-9752-4d63-a878-129a34a0647b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:16.597319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d565e8-9752-4d63-a878-129a34a0647b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:16.597764 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d565e8-9752-4d63-a878-129a34a0647b admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:16.598186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d565e8-9752-4d63-a878-129a34a0647b admin admin] PATH_INFO: `/v3/users/2710c46896c642dca92b3cfe2ea4bf8f` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:16.599635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d565e8-9752-4d63-a878-129a34a0647b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2710c46896c642dca92b3cfe2ea4bf8f)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:16.601320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb1f23-ee5e-4e69-b2a0-5509a4a05a33 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:16.602933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d565e8-9752-4d63-a878-129a34a0647b admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:16.607066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5f6be-921a-4b2e-9983-925483375320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:16.610747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5f6be-921a-4b2e-9983-925483375320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:16.610747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5f6be-921a-4b2e-9983-925483375320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:16.610747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5f6be-921a-4b2e-9983-925483375320 admin admin] PATH_INFO: `/v3/users/163e176baa1e479ba03f94d0e5e422c3` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:16.610747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5f6be-921a-4b2e-9983-925483375320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=163e176baa1e479ba03f94d0e5e422c3)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:16.613574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5f6be-921a-4b2e-9983-925483375320 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:16.911517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f8d565e8-9752-4d63-a878-129a34a0647b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2710c46896c642dca92b3cfe2ea4bf8f'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:16.912443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f8d565e8-9752-4d63-a878-129a34a0647b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2710c46896c642dca92b3cfe2ea4bf8f'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:16.930201 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-adb5f6be-921a-4b2e-9983-925483375320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '163e176baa1e479ba03f94d0e5e422c3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:17.023796 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f8d565e8-9752-4d63-a878-129a34a0647b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2710c46896c642dca92b3cfe2ea4bf8f'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:17.073380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f8d565e8-9752-4d63-a878-129a34a0647b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2710c46896c642dca92b3cfe2ea4bf8f'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:17.105940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f8d565e8-9752-4d63-a878-129a34a0647b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2710c46896c642dca92b3cfe2ea4bf8f'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:17.118845 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 866/1743] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:16 2022] DELETE /identity/v3/users/2710c46896c642dca92b3cfe2ea4bf8f => generated 0 bytes in 817 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:17.120556 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50b95a-fac7-480b-9f4a-74492a5e04c5 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:17.121120 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50b95a-fac7-480b-9f4a-74492a5e04c5 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:17.121610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50b95a-fac7-480b-9f4a-74492a5e04c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:17.137666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-adb5f6be-921a-4b2e-9983-925483375320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '163e176baa1e479ba03f94d0e5e422c3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:17.139711 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-adb5f6be-921a-4b2e-9983-925483375320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '163e176baa1e479ba03f94d0e5e422c3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:17.158446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-adb5f6be-921a-4b2e-9983-925483375320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '163e176baa1e479ba03f94d0e5e422c3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:17.183775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-9b50b95a-fac7-480b-9f4a-74492a5e04c5 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:17.188341 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9b50b95a-fac7-480b-9f4a-74492a5e04c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:17.203159 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-adb5f6be-921a-4b2e-9983-925483375320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '163e176baa1e479ba03f94d0e5e422c3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:17.214256 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 878/1744] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:16 2022] DELETE /identity/v3/users/163e176baa1e479ba03f94d0e5e422c3 => generated 0 bytes in 767 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:17.214256 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5f6be-921a-4b2e-9983-925483375320 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:17.235221 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-adb5f6be-921a-4b2e-9983-925483375320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:17.423978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 867/1745] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:17.425988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c478a7f-ab16-440d-9897-2fc5c2813d12 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:17.428439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c478a7f-ab16-440d-9897-2fc5c2813d12 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:17.429024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c478a7f-ab16-440d-9897-2fc5c2813d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:17.444038 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5f6be-921a-4b2e-9983-925483375320 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:17.450867 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:17.469081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6658e6c-1cf0-4509-856f-90d5983acd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:17.469743 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6658e6c-1cf0-4509-856f-90d5983acd05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:17.470098 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6658e6c-1cf0-4509-856f-90d5983acd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:17.470379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6658e6c-1cf0-4509-856f-90d5983acd05 admin admin] PATH_INFO: `/v3/projects/d526623066fb41aa8c2d8e09e7b6858c` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:17.473623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6658e6c-1cf0-4509-856f-90d5983acd05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d526623066fb41aa8c2d8e09e7b6858c)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:17.483307 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6658e6c-1cf0-4509-856f-90d5983acd05 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:17.483870 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-8c478a7f-ab16-440d-9897-2fc5c2813d12 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:17.488091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8c478a7f-ab16-440d-9897-2fc5c2813d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:17.507767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 868/1746] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:17.510209 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbaee99-9a09-45f7-83d6-b9f024b3972b None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:17.512936 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbaee99-9a09-45f7-83d6-b9f024b3972b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:17.513219 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbaee99-9a09-45f7-83d6-b9f024b3972b None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:17.514947 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 869/1747] 10.222.0.27 () {54 vars in 921 bytes} [Fri Aug 12 12:19:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:19:17.519146 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa5695-6cd8-4c74-9812-b45014b39607 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:17.519433 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa5695-6cd8-4c74-9812-b45014b39607 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:17.519716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa5695-6cd8-4c74-9812-b45014b39607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:17.576186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-0efa5695-6cd8-4c74-9812-b45014b39607 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:17.577902 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0efa5695-6cd8-4c74-9812-b45014b39607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:17.617033 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 870/1748] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:17.618743 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60354df1-2735-448d-9329-7cc6e353dce5 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:17.619036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60354df1-2735-448d-9329-7cc6e353dce5 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:17.619388 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60354df1-2735-448d-9329-7cc6e353dce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:17.650474 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-60354df1-2735-448d-9329-7cc6e353dce5 None None] MFA Rules not processed for user `687d0375abd44d2481f988c6df90fad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:17.651942 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-60354df1-2735-448d-9329-7cc6e353dce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:17.693790 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b6658e6c-1cf0-4509-856f-90d5983acd05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd526623066fb41aa8c2d8e09e7b6858c', 'request_id': 'req-b6658e6c-1cf0-4509-856f-90d5983acd05'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:17.707176 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b6658e6c-1cf0-4509-856f-90d5983acd05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd526623066fb41aa8c2d8e09e7b6858c', 'request_id': 'req-b6658e6c-1cf0-4509-856f-90d5983acd05'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:17.722940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b6658e6c-1cf0-4509-856f-90d5983acd05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd526623066fb41aa8c2d8e09e7b6858c', 'request_id': 'req-b6658e6c-1cf0-4509-856f-90d5983acd05'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:17.765917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 871/1749] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Aug 12 12:19:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:17.767366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60354df1-2735-448d-9329-7cc6e353dce5 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:17.768523 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-60354df1-2735-448d-9329-7cc6e353dce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:17.816680 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b6658e6c-1cf0-4509-856f-90d5983acd05 admin admin] The token cache is being invalidate because project d526623066fb41aa8c2d8e09e7b6858c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:17.817193 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b6658e6c-1cf0-4509-856f-90d5983acd05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:17.818336 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b6658e6c-1cf0-4509-856f-90d5983acd05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:17.820331 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 879/1750] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:17 2022] DELETE /identity/v3/projects/d526623066fb41aa8c2d8e09e7b6858c => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:17.821529 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6658e6c-1cf0-4509-856f-90d5983acd05 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:17.830162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b6658e6c-1cf0-4509-856f-90d5983acd05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:17.992730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60354df1-2735-448d-9329-7cc6e353dce5 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:18.006958 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05800c9-2dc4-4cde-94aa-d1d3a797ef5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:18.007710 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05800c9-2dc4-4cde-94aa-d1d3a797ef5f service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:18.008005 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05800c9-2dc4-4cde-94aa-d1d3a797ef5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:18.008285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05800c9-2dc4-4cde-94aa-d1d3a797ef5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:18.018037 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c05800c9-2dc4-4cde-94aa-d1d3a797ef5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:18.165600 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6658e6c-1cf0-4509-856f-90d5983acd05 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:18.176039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7a4c9-3429-4c8d-8dd8-6f267f83d2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:18.185448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7a4c9-3429-4c8d-8dd8-6f267f83d2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:18.185448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7a4c9-3429-4c8d-8dd8-6f267f83d2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:18.185448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7a4c9-3429-4c8d-8dd8-6f267f83d2b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:18.185448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7a4c9-3429-4c8d-8dd8-6f267f83d2b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:19:18.185448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7a4c9-3429-4c8d-8dd8-6f267f83d2b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:18.185448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7a4c9-3429-4c8d-8dd8-6f267f83d2b7 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:18.209505 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05800c9-2dc4-4cde-94aa-d1d3a797ef5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:18.211805 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05800c9-2dc4-4cde-94aa-d1d3a797ef5f service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:18.232686 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 880/1751] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:19:18.235178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f6f1e-b194-4fa9-afa0-c221dd86b12d None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:18.235900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f6f1e-b194-4fa9-afa0-c221dd86b12d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:18.236688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f6f1e-b194-4fa9-afa0-c221dd86b12d None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:18.240152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 881/1752] 10.222.0.27 () {54 vars in 921 bytes} [Fri Aug 12 12:19:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:19:18.242158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f40524-3648-4d24-9a69-e95d7f25b123 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:18.242515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f40524-3648-4d24-9a69-e95d7f25b123 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:18.242772 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f40524-3648-4d24-9a69-e95d7f25b123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:18.250789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 872/1753] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:18.254780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05800c9-2dc4-4cde-94aa-d1d3a797ef5f service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:18.288330 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05800c9-2dc4-4cde-94aa-d1d3a797ef5f service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:18.296056 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f0f40524-3648-4d24-9a69-e95d7f25b123 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:18.300235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f0f40524-3648-4d24-9a69-e95d7f25b123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:18.311557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 882/1754] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:18.312849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:18.325481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f8958-d8fe-40cd-8759-b0bfdd11bbf2 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:18.325481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f8958-d8fe-40cd-8759-b0bfdd11bbf2 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:18.325481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f8958-d8fe-40cd-8759-b0bfdd11bbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:18.334060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643a5ad-ea23-4612-9054-7755b8dc156b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:18.334914 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643a5ad-ea23-4612-9054-7755b8dc156b admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:18.335204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643a5ad-ea23-4612-9054-7755b8dc156b admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:18.335520 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643a5ad-ea23-4612-9054-7755b8dc156b admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:18.336412 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643a5ad-ea23-4612-9054-7755b8dc156b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:18.339108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643a5ad-ea23-4612-9054-7755b8dc156b admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:18.451807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:18.452585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 873/1755] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:18 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:18.457334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7938f-dea4-4587-a8c7-8568947c9611 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:18.457925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7938f-dea4-4587-a8c7-8568947c9611 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:18.459663 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7938f-dea4-4587-a8c7-8568947c9611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:18.475013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-c04f8958-d8fe-40cd-8759-b0bfdd11bbf2 None None] MFA Rules not processed for user `687d0375abd44d2481f988c6df90fad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:18.475234 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-c04f8958-d8fe-40cd-8759-b0bfdd11bbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:18.505532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-74f7938f-dea4-4587-a8c7-8568947c9611 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:18.509660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-74f7938f-dea4-4587-a8c7-8568947c9611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:18.658717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 874/1756] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:18.659862 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7938f-dea4-4587-a8c7-8568947c9611 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:18.672766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 883/1757] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Aug 12 12:19:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:18.674259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f8958-d8fe-40cd-8759-b0bfdd11bbf2 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:18.685696 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7938f-dea4-4587-a8c7-8568947c9611 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:18.693351 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ebabd-f249-4f62-8b8e-3724aabe055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:18.694156 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ebabd-f249-4f62-8b8e-3724aabe055e admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:18.695310 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ebabd-f249-4f62-8b8e-3724aabe055e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:18.695744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ebabd-f249-4f62-8b8e-3724aabe055e admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:18.696959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ebabd-f249-4f62-8b8e-3724aabe055e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:18.699501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ebabd-f249-4f62-8b8e-3724aabe055e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:18.704213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f8958-d8fe-40cd-8759-b0bfdd11bbf2 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:18.730192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de12bc6-fded-45db-9e29-43d007c64612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:18.730739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de12bc6-fded-45db-9e29-43d007c64612 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:18.731088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de12bc6-fded-45db-9e29-43d007c64612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:18.731454 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de12bc6-fded-45db-9e29-43d007c64612 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:18.732065 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de12bc6-fded-45db-9e29-43d007c64612 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:19:18.732429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de12bc6-fded-45db-9e29-43d007c64612 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:18.747800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de12bc6-fded-45db-9e29-43d007c64612 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:18.794978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 884/1758] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:19:18.794978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de12bc6-fded-45db-9e29-43d007c64612 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:18.799217 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4de12bc6-fded-45db-9e29-43d007c64612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:18.829221 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de12bc6-fded-45db-9e29-43d007c64612 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:18.836468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9cb7b-ec52-4b79-8a93-a86a59611dcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:18.837951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9cb7b-ec52-4b79-8a93-a86a59611dcc service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:18.837951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9cb7b-ec52-4b79-8a93-a86a59611dcc service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:18.838220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9cb7b-ec52-4b79-8a93-a86a59611dcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:18.840500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f4d9cb7b-ec52-4b79-8a93-a86a59611dcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:18.850584 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 875/1759] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:18 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:18.850976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ebabd-f249-4f62-8b8e-3724aabe055e admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:18.856041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1e0ebabd-f249-4f62-8b8e-3724aabe055e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:18.858667 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9cb7b-ec52-4b79-8a93-a86a59611dcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:18.861576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9cb7b-ec52-4b79-8a93-a86a59611dcc service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:18.886731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 885/1760] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:18.886731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9cb7b-ec52-4b79-8a93-a86a59611dcc service nova] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:18.907993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ebabd-f249-4f62-8b8e-3724aabe055e admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:18.917864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9cb7b-ec52-4b79-8a93-a86a59611dcc service nova] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:18.924072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08b8d5-70ca-46ae-a5c4-53e6dd69ab24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:18.928135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08b8d5-70ca-46ae-a5c4-53e6dd69ab24 service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:18.929443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08b8d5-70ca-46ae-a5c4-53e6dd69ab24 service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:18.930707 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08b8d5-70ca-46ae-a5c4-53e6dd69ab24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:18.935923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a954c4-90c2-4b10-b9da-a60c533450be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:18.937166 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a954c4-90c2-4b10-b9da-a60c533450be admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:18.938242 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a954c4-90c2-4b10-b9da-a60c533450be admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:18.939983 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a954c4-90c2-4b10-b9da-a60c533450be admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:18.940840 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a954c4-90c2-4b10-b9da-a60c533450be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:18.946354 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a954c4-90c2-4b10-b9da-a60c533450be admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:18.967978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08b8d5-70ca-46ae-a5c4-53e6dd69ab24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:18.975393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08b8d5-70ca-46ae-a5c4-53e6dd69ab24 service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:19.015875 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 876/1761] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:19.018024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08b8d5-70ca-46ae-a5c4-53e6dd69ab24 service nova] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:19.043055 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08b8d5-70ca-46ae-a5c4-53e6dd69ab24 service nova] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:19.057714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d5e55-c833-4848-8243-964851d1753b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:19.058303 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d5e55-c833-4848-8243-964851d1753b admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:19.058583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d5e55-c833-4848-8243-964851d1753b admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:19.058914 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d5e55-c833-4848-8243-964851d1753b admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:19.059564 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d5e55-c833-4848-8243-964851d1753b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:19.063376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d5e55-c833-4848-8243-964851d1753b admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:19.099314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 886/1762] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:18 2022] POST /identity/v3/projects => generated 381 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:19.100293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a954c4-90c2-4b10-b9da-a60c533450be admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:19.103699 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 877/1763] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:19.110141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d5e55-c833-4848-8243-964851d1753b admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:19.127622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-860d5e55-c833-4848-8243-964851d1753b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:19.217288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d5e55-c833-4848-8243-964851d1753b admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:19.271083 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a954c4-90c2-4b10-b9da-a60c533450be admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:19.282347 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b1214-0824-4923-8fae-0f324786dac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:19.286944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b1214-0824-4923-8fae-0f324786dac6 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:19.286944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b1214-0824-4923-8fae-0f324786dac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:19.286944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b1214-0824-4923-8fae-0f324786dac6 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:19.286944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b1214-0824-4923-8fae-0f324786dac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:19.303919 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b1214-0824-4923-8fae-0f324786dac6 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:19.413677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5b361-428f-4b28-8420-2cfb0d77a0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:19.414583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5b361-428f-4b28-8420-2cfb0d77a0e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:19.415062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5b361-428f-4b28-8420-2cfb0d77a0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:19.415699 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5b361-428f-4b28-8420-2cfb0d77a0e4 admin admin] PATH_INFO: `/v3/projects/8ae584646651473f9cabd9309808115a` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:19.417717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5b361-428f-4b28-8420-2cfb0d77a0e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae584646651473f9cabd9309808115a)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:19.420096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5b361-428f-4b28-8420-2cfb0d77a0e4 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:19.426984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 887/1764] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:19 2022] POST /identity/v3/users => generated 291 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:19.431136 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b1214-0824-4923-8fae-0f324786dac6 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:19.460450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b1214-0824-4923-8fae-0f324786dac6 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:19.478496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c53930-3f7f-4e2d-9592-61c7758db779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:19.478959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c53930-3f7f-4e2d-9592-61c7758db779 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:19.479422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c53930-3f7f-4e2d-9592-61c7758db779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:19.479862 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c53930-3f7f-4e2d-9592-61c7758db779 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:19.480925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c53930-3f7f-4e2d-9592-61c7758db779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:19.482744 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c53930-3f7f-4e2d-9592-61c7758db779 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:19.500353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 888/1765] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:19.500353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c53930-3f7f-4e2d-9592-61c7758db779 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:19.523475 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c53930-3f7f-4e2d-9592-61c7758db779 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:19.544944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85091d6-f72e-43d6-893e-1395374c637b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:19.550153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85091d6-f72e-43d6-893e-1395374c637b admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:19.551059 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85091d6-f72e-43d6-893e-1395374c637b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:19.556304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85091d6-f72e-43d6-893e-1395374c637b admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:19.556304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85091d6-f72e-43d6-893e-1395374c637b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:19.556304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85091d6-f72e-43d6-893e-1395374c637b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:19.584705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 889/1766] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:19.584705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85091d6-f72e-43d6-893e-1395374c637b admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:19.593250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-27b5b361-428f-4b28-8420-2cfb0d77a0e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae584646651473f9cabd9309808115a', 'request_id': 'req-27b5b361-428f-4b28-8420-2cfb0d77a0e4'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:19.602323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-27b5b361-428f-4b28-8420-2cfb0d77a0e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae584646651473f9cabd9309808115a', 'request_id': 'req-27b5b361-428f-4b28-8420-2cfb0d77a0e4'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:19.608422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85091d6-f72e-43d6-893e-1395374c637b admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:19.612097 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:19.615022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-27b5b361-428f-4b28-8420-2cfb0d77a0e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae584646651473f9cabd9309808115a', 'request_id': 'req-27b5b361-428f-4b28-8420-2cfb0d77a0e4'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:19.634703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:19.695405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-27b5b361-428f-4b28-8420-2cfb0d77a0e4 admin admin] The token cache is being invalidate because project 8ae584646651473f9cabd9309808115a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:19.696022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-27b5b361-428f-4b28-8420-2cfb0d77a0e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:19.696968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-27b5b361-428f-4b28-8420-2cfb0d77a0e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:19.698939 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 878/1767] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:19 2022] DELETE /identity/v3/projects/8ae584646651473f9cabd9309808115a => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:19.702436 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5b361-428f-4b28-8420-2cfb0d77a0e4 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:19.705558 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-27b5b361-428f-4b28-8420-2cfb0d77a0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:19.739490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd8d4e-d146-49ab-aa87-1ca1a89bae7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:19.740462 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd8d4e-d146-49ab-aa87-1ca1a89bae7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:19.740903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd8d4e-d146-49ab-aa87-1ca1a89bae7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:19.741453 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd8d4e-d146-49ab-aa87-1ca1a89bae7e admin admin] PATH_INFO: `/v3/projects/8865934723c34e17b1237525a10f28ff/users/19511af5527940daaf50aaf01dd5ce43/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:19.758674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd8d4e-d146-49ab-aa87-1ca1a89bae7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8865934723c34e17b1237525a10f28ff, user_id=19511af5527940daaf50aaf01dd5ce43, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:19.761561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd8d4e-d146-49ab-aa87-1ca1a89bae7e admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:19.843484 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5b361-428f-4b28-8420-2cfb0d77a0e4 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:19.853566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed895c-5d8c-4ce2-9e43-d8e39979a1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:19.854053 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed895c-5d8c-4ce2-9e43-d8e39979a1ba admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:19.862468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 890/1768] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:19 2022] PUT /identity/v3/projects/8865934723c34e17b1237525a10f28ff/users/19511af5527940daaf50aaf01dd5ce43/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:19.862608 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed895c-5d8c-4ce2-9e43-d8e39979a1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:19.862608 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed895c-5d8c-4ce2-9e43-d8e39979a1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:19.862608 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed895c-5d8c-4ce2-9e43-d8e39979a1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:19.862860 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d7f6c-1c66-4e0e-a2d3-bb819b45bb6c None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:19.862860 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d7f6c-1c66-4e0e-a2d3-bb819b45bb6c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:19.862860 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d7f6c-1c66-4e0e-a2d3-bb819b45bb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:19.863082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed895c-5d8c-4ce2-9e43-d8e39979a1ba admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:19.888803 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 879/1769] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:19.895309 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89991c-e688-4928-a830-f82b11d72eb5 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:19.895309 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89991c-e688-4928-a830-f82b11d72eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:19.895309 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89991c-e688-4928-a830-f82b11d72eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:19.911804 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f90d7f6c-1c66-4e0e-a2d3-bb819b45bb6c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:19.913123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f90d7f6c-1c66-4e0e-a2d3-bb819b45bb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:19.967690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-ea89991c-e688-4928-a830-f82b11d72eb5 None None] MFA Rules not processed for user `19511af5527940daaf50aaf01dd5ce43`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:19.969218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-ea89991c-e688-4928-a830-f82b11d72eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:20.020190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 891/1770] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:20.021133 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d7f6c-1c66-4e0e-a2d3-bb819b45bb6c None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:20.049643 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d7f6c-1c66-4e0e-a2d3-bb819b45bb6c None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:20.057164 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f972e1-4510-40dc-b346-8d104ed382d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:20.061536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f972e1-4510-40dc-b346-8d104ed382d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:20.061536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f972e1-4510-40dc-b346-8d104ed382d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:20.061536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f972e1-4510-40dc-b346-8d104ed382d3 admin admin] PATH_INFO: `/v3/projects/36f63c08eab1497f80d314e496bc5be9/users/25cb9992166e4464a72fd0741edd9bb6/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:20.102802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f972e1-4510-40dc-b346-8d104ed382d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36f63c08eab1497f80d314e496bc5be9, user_id=25cb9992166e4464a72fd0741edd9bb6, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:20.113419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f972e1-4510-40dc-b346-8d104ed382d3 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:20.130473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 880/1771] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:20.132430 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b8df0-91a3-430c-b923-50a2b2c0dbd1 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:20.133002 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b8df0-91a3-430c-b923-50a2b2c0dbd1 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:20.134262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b8df0-91a3-430c-b923-50a2b2c0dbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:20.169492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-a69b8df0-91a3-430c-b923-50a2b2c0dbd1 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:20.173890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a69b8df0-91a3-430c-b923-50a2b2c0dbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:20.176350 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:20.187971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 881/1772] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:20.189663 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b8df0-91a3-430c-b923-50a2b2c0dbd1 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:20.192589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a69b8df0-91a3-430c-b923-50a2b2c0dbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:20.209585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b8df0-91a3-430c-b923-50a2b2c0dbd1 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:20.214775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 892/1773] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:20 2022] PUT /identity/v3/projects/36f63c08eab1497f80d314e496bc5be9/users/25cb9992166e4464a72fd0741edd9bb6/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:20.218022 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f972e1-4510-40dc-b346-8d104ed382d3 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:20.349659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f972e1-4510-40dc-b346-8d104ed382d3 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:20.364719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:20.378818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871f563-459a-463d-9f36-eb4380e3d47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:20.379462 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871f563-459a-463d-9f36-eb4380e3d47d admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:20.379787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871f563-459a-463d-9f36-eb4380e3d47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:20.380032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871f563-459a-463d-9f36-eb4380e3d47d admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:20.380748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871f563-459a-463d-9f36-eb4380e3d47d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:19:20.381088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871f563-459a-463d-9f36-eb4380e3d47d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:20.383610 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871f563-459a-463d-9f36-eb4380e3d47d admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:20.421719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 893/1774] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:19:20.421719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a35b7d-54dd-404b-98e5-8f176eded7fd None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:20.421719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a35b7d-54dd-404b-98e5-8f176eded7fd None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:20.421719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a35b7d-54dd-404b-98e5-8f176eded7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:20.474616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425cb20-47f4-4a75-98e7-9cd4d2bee850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:20.474616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425cb20-47f4-4a75-98e7-9cd4d2bee850 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:20.474616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425cb20-47f4-4a75-98e7-9cd4d2bee850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:20.474616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425cb20-47f4-4a75-98e7-9cd4d2bee850 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:20.475743 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425cb20-47f4-4a75-98e7-9cd4d2bee850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:20.475743 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425cb20-47f4-4a75-98e7-9cd4d2bee850 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:20.489208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-94a35b7d-54dd-404b-98e5-8f176eded7fd None None] MFA Rules not processed for user `25cb9992166e4464a72fd0741edd9bb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:20.491493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-94a35b7d-54dd-404b-98e5-8f176eded7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:20.636615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 882/1775] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:20 2022] POST /identity/v3/projects => generated 387 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:20.636615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425cb20-47f4-4a75-98e7-9cd4d2bee850 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:20.642069 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a425cb20-47f4-4a75-98e7-9cd4d2bee850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:20.657416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 894/1776] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:20.657683 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a35b7d-54dd-404b-98e5-8f176eded7fd None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:20.670437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a35b7d-54dd-404b-98e5-8f176eded7fd None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:20.831957 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508722e-30c1-4dd5-8c87-3ddbf370e588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:20.841040 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508722e-30c1-4dd5-8c87-3ddbf370e588 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:20.841631 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508722e-30c1-4dd5-8c87-3ddbf370e588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:20.842260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508722e-30c1-4dd5-8c87-3ddbf370e588 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:20.843351 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508722e-30c1-4dd5-8c87-3ddbf370e588 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:20.847148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508722e-30c1-4dd5-8c87-3ddbf370e588 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:20.857121 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425cb20-47f4-4a75-98e7-9cd4d2bee850 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:20.865268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf313c-51e8-4863-84b3-b858bf401203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:20.869316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf313c-51e8-4863-84b3-b858bf401203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:20.869316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf313c-51e8-4863-84b3-b858bf401203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:20.869316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf313c-51e8-4863-84b3-b858bf401203 admin admin] PATH_INFO: `/v3/projects/bacf73e4c9714e7daa0b61af79f767c5` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:20.869316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf313c-51e8-4863-84b3-b858bf401203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bacf73e4c9714e7daa0b61af79f767c5)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:20.881352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf313c-51e8-4863-84b3-b858bf401203 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:20.978426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 895/1777] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:20 2022] POST /identity/v3/projects => generated 375 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:20.979532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508722e-30c1-4dd5-8c87-3ddbf370e588 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:21.072588 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-53bf313c-51e8-4863-84b3-b858bf401203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bacf73e4c9714e7daa0b61af79f767c5', 'request_id': 'req-53bf313c-51e8-4863-84b3-b858bf401203'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:21.092366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-53bf313c-51e8-4863-84b3-b858bf401203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bacf73e4c9714e7daa0b61af79f767c5', 'request_id': 'req-53bf313c-51e8-4863-84b3-b858bf401203'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:21.117418 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-53bf313c-51e8-4863-84b3-b858bf401203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bacf73e4c9714e7daa0b61af79f767c5', 'request_id': 'req-53bf313c-51e8-4863-84b3-b858bf401203'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:21.157963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508722e-30c1-4dd5-8c87-3ddbf370e588 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:21.182833 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:21.227830 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-53bf313c-51e8-4863-84b3-b858bf401203 admin admin] The token cache is being invalidate because project bacf73e4c9714e7daa0b61af79f767c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:21.228141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-53bf313c-51e8-4863-84b3-b858bf401203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:21.228827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-53bf313c-51e8-4863-84b3-b858bf401203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:21.231905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 883/1778] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:20 2022] DELETE /identity/v3/projects/bacf73e4c9714e7daa0b61af79f767c5 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:21.235916 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf313c-51e8-4863-84b3-b858bf401203 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:21.240030 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-53bf313c-51e8-4863-84b3-b858bf401203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:21.311905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fbd9a-4657-48b6-9fce-f040232b6e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:21.317091 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fbd9a-4657-48b6-9fce-f040232b6e93 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:21.317671 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fbd9a-4657-48b6-9fce-f040232b6e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:21.318157 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fbd9a-4657-48b6-9fce-f040232b6e93 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:21.319461 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fbd9a-4657-48b6-9fce-f040232b6e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:21.325567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fbd9a-4657-48b6-9fce-f040232b6e93 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:21.357664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf313c-51e8-4863-84b3-b858bf401203 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:21.364388 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b959a-7459-44e2-98d0-94c32566277c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:21.365647 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b959a-7459-44e2-98d0-94c32566277c admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:21.366507 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b959a-7459-44e2-98d0-94c32566277c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:21.404564 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b959a-7459-44e2-98d0-94c32566277c admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:21.405302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b959a-7459-44e2-98d0-94c32566277c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:21.417929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b959a-7459-44e2-98d0-94c32566277c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:21.450330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 896/1779] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:20 2022] POST /identity/v3/users => generated 294 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:21.450625 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fbd9a-4657-48b6-9fce-f040232b6e93 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:21.458596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-c42fbd9a-4657-48b6-9fce-f040232b6e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:21.523341 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fbd9a-4657-48b6-9fce-f040232b6e93 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:21.544332 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb64b4-7801-438e-8094-2ed0460e1673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:21.548613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb64b4-7801-438e-8094-2ed0460e1673 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:21.550405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb64b4-7801-438e-8094-2ed0460e1673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:21.550798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb64b4-7801-438e-8094-2ed0460e1673 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:21.553640 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb64b4-7801-438e-8094-2ed0460e1673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:21.560774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb64b4-7801-438e-8094-2ed0460e1673 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:21.594509 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 884/1780] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:21 2022] POST /identity/v3/projects => generated 381 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:21.594509 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b959a-7459-44e2-98d0-94c32566277c admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:21.599372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-dc9b959a-7459-44e2-98d0-94c32566277c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:21.700133 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 897/1781] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:21 2022] POST /identity/v3/users => generated 288 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:21.701312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb64b4-7801-438e-8094-2ed0460e1673 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:21.714382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b9bb64b4-7801-438e-8094-2ed0460e1673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:21.749237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb64b4-7801-438e-8094-2ed0460e1673 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:21.762979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e541c-b8ad-4b37-8ae8-574bd574cbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:21.763937 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:21.765357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e541c-b8ad-4b37-8ae8-574bd574cbf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:21.770718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e541c-b8ad-4b37-8ae8-574bd574cbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:21.770718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e541c-b8ad-4b37-8ae8-574bd574cbf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:21.770718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e541c-b8ad-4b37-8ae8-574bd574cbf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:21.770964 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e541c-b8ad-4b37-8ae8-574bd574cbf9 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:21.772708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b959a-7459-44e2-98d0-94c32566277c admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:21.792083 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 898/1782] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:21.792972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3d547-31df-4861-a26b-16a2e8cb038e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:21.793696 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3d547-31df-4861-a26b-16a2e8cb038e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:21.793953 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e541c-b8ad-4b37-8ae8-574bd574cbf9 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:21.799296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3d547-31df-4861-a26b-16a2e8cb038e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:21.799296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3d547-31df-4861-a26b-16a2e8cb038e admin admin] PATH_INFO: `/v3/users/da67d8dc151245158cde6b96c3cfa754` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:21.799296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3d547-31df-4861-a26b-16a2e8cb038e admin admin] RBAC: Authorizing `identity:delete_user(user_id=da67d8dc151245158cde6b96c3cfa754)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:21.811712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3d547-31df-4861-a26b-16a2e8cb038e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:21.812290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e541c-b8ad-4b37-8ae8-574bd574cbf9 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:21.820592 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7e627-a3a6-4ae2-a9f4-0575d730f6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:21.821373 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7e627-a3a6-4ae2-a9f4-0575d730f6fb admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:21.821744 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7e627-a3a6-4ae2-a9f4-0575d730f6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:21.822078 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7e627-a3a6-4ae2-a9f4-0575d730f6fb admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:21.822790 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7e627-a3a6-4ae2-a9f4-0575d730f6fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:21.826596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7e627-a3a6-4ae2-a9f4-0575d730f6fb admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:21.965152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 899/1783] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:21 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:21.967637 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7e627-a3a6-4ae2-a9f4-0575d730f6fb admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:22.002787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7e627-a3a6-4ae2-a9f4-0575d730f6fb admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:22.017802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b0ea7-6b3b-4751-b3e2-5291ae7a342a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:22.027177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b0ea7-6b3b-4751-b3e2-5291ae7a342a admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:22.027177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b0ea7-6b3b-4751-b3e2-5291ae7a342a admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:22.027177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b0ea7-6b3b-4751-b3e2-5291ae7a342a admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:22.027177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b0ea7-6b3b-4751-b3e2-5291ae7a342a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:22.027177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b0ea7-6b3b-4751-b3e2-5291ae7a342a admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:22.068423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 900/1784] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:22.069799 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b0ea7-6b3b-4751-b3e2-5291ae7a342a admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:22.085357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b0ea7-6b3b-4751-b3e2-5291ae7a342a admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:22.092228 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd339f-23c1-4d37-8877-bc0e9cbdb713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:22.093641 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd339f-23c1-4d37-8877-bc0e9cbdb713 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:22.094981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd339f-23c1-4d37-8877-bc0e9cbdb713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:22.095613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd339f-23c1-4d37-8877-bc0e9cbdb713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:22.096896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd339f-23c1-4d37-8877-bc0e9cbdb713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:22.099951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd339f-23c1-4d37-8877-bc0e9cbdb713 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:22.123444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 901/1785] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:22.124610 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd339f-23c1-4d37-8877-bc0e9cbdb713 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:22.134850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-21a3d547-31df-4861-a26b-16a2e8cb038e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da67d8dc151245158cde6b96c3cfa754'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:22.135817 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-21a3d547-31df-4861-a26b-16a2e8cb038e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da67d8dc151245158cde6b96c3cfa754'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:22.144388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd339f-23c1-4d37-8877-bc0e9cbdb713 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:22.149666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091158e-5019-4ad5-82ef-5389c4dbffaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:22.150333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091158e-5019-4ad5-82ef-5389c4dbffaa admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:22.150869 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091158e-5019-4ad5-82ef-5389c4dbffaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:22.151074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091158e-5019-4ad5-82ef-5389c4dbffaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:22.151816 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091158e-5019-4ad5-82ef-5389c4dbffaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:22.153534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091158e-5019-4ad5-82ef-5389c4dbffaa admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:22.182133 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 902/1786] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:22.186073 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091158e-5019-4ad5-82ef-5389c4dbffaa admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:22.188927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e091158e-5019-4ad5-82ef-5389c4dbffaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:22.211581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091158e-5019-4ad5-82ef-5389c4dbffaa admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:22.222113 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39fb4b-1d12-4c03-905b-47f0249468ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:22.222747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39fb4b-1d12-4c03-905b-47f0249468ee admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:22.223033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39fb4b-1d12-4c03-905b-47f0249468ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:22.223316 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39fb4b-1d12-4c03-905b-47f0249468ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:22.224001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39fb4b-1d12-4c03-905b-47f0249468ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:22.225628 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39fb4b-1d12-4c03-905b-47f0249468ee admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:22.227769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-21a3d547-31df-4861-a26b-16a2e8cb038e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da67d8dc151245158cde6b96c3cfa754'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:22.256543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 903/1787] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:22.257458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39fb4b-1d12-4c03-905b-47f0249468ee admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:22.261001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-bf39fb4b-1d12-4c03-905b-47f0249468ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:22.283905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-21a3d547-31df-4861-a26b-16a2e8cb038e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da67d8dc151245158cde6b96c3cfa754'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:22.288982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39fb4b-1d12-4c03-905b-47f0249468ee admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:22.302004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d23a0a-5a18-414d-9fea-327e0eafcba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:22.302710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d23a0a-5a18-414d-9fea-327e0eafcba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:22.303112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d23a0a-5a18-414d-9fea-327e0eafcba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:22.303384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d23a0a-5a18-414d-9fea-327e0eafcba8 admin admin] PATH_INFO: `/v3/projects/35f34cd06c8e49f68ceac844d2cc528f/users/c086e8fa2f8141b29a3d8b35272c7f3c/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:22.314729 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-21a3d547-31df-4861-a26b-16a2e8cb038e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da67d8dc151245158cde6b96c3cfa754'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:22.323034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 885/1788] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:21 2022] DELETE /identity/v3/users/da67d8dc151245158cde6b96c3cfa754 => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:22.324480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3d547-31df-4861-a26b-16a2e8cb038e admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:22.327780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-21a3d547-31df-4861-a26b-16a2e8cb038e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:22.332831 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d23a0a-5a18-414d-9fea-327e0eafcba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35f34cd06c8e49f68ceac844d2cc528f, user_id=c086e8fa2f8141b29a3d8b35272c7f3c, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:22.336614 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d23a0a-5a18-414d-9fea-327e0eafcba8 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:22.378635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:22.420346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 904/1789] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:22 2022] PUT /identity/v3/projects/35f34cd06c8e49f68ceac844d2cc528f/users/c086e8fa2f8141b29a3d8b35272c7f3c/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:22.421489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d23a0a-5a18-414d-9fea-327e0eafcba8 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:22.445050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3d547-31df-4861-a26b-16a2e8cb038e admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:22.560862 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7f1f6-6b21-4956-ae3f-7d846122beb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:22.561705 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7f1f6-6b21-4956-ae3f-7d846122beb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:22.562084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7f1f6-6b21-4956-ae3f-7d846122beb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:22.562517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7f1f6-6b21-4956-ae3f-7d846122beb1 admin admin] PATH_INFO: `/v3/projects/fe1e0cd3163c42f0b2a42d7a0ebd8d60/users/bb28fbe8b93e4d5686800dd17b283085/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:22.576231 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d23a0a-5a18-414d-9fea-327e0eafcba8 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:22.585241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7f1f6-6b21-4956-ae3f-7d846122beb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1e0cd3163c42f0b2a42d7a0ebd8d60, user_id=bb28fbe8b93e4d5686800dd17b283085, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:22.588244 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7f1f6-6b21-4956-ae3f-7d846122beb1 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:22.598633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:22.603320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb362cc-619b-49ca-ac61-a08a30698fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:22.604026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb362cc-619b-49ca-ac61-a08a30698fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:22.605238 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb362cc-619b-49ca-ac61-a08a30698fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:22.605238 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb362cc-619b-49ca-ac61-a08a30698fa3 admin admin] PATH_INFO: `/v3/projects/4fdd6741d3694ceda2655dc72b5cd75e/users/b88dd5b396354ff9a4effb30e4ac8572/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:22.658629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 886/1790] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:22 2022] PUT /identity/v3/projects/fe1e0cd3163c42f0b2a42d7a0ebd8d60/users/bb28fbe8b93e4d5686800dd17b283085/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:22.662551 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb362cc-619b-49ca-ac61-a08a30698fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fdd6741d3694ceda2655dc72b5cd75e, user_id=b88dd5b396354ff9a4effb30e4ac8572, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:22.662673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2885f22-ba34-48f4-825d-8fccb552fd5a None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:22.662673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2885f22-ba34-48f4-825d-8fccb552fd5a None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:22.662673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2885f22-ba34-48f4-825d-8fccb552fd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:22.666109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb362cc-619b-49ca-ac61-a08a30698fa3 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:22.723015 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-b2885f22-ba34-48f4-825d-8fccb552fd5a None None] MFA Rules not processed for user `c086e8fa2f8141b29a3d8b35272c7f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:22.724615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b2885f22-ba34-48f4-825d-8fccb552fd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:22.746000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 905/1791] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:22 2022] PUT /identity/v3/projects/4fdd6741d3694ceda2655dc72b5cd75e/users/b88dd5b396354ff9a4effb30e4ac8572/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:22.748158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb362cc-619b-49ca-ac61-a08a30698fa3 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:22.842880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb362cc-619b-49ca-ac61-a08a30698fa3 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:22.855403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1f308-c325-4916-88ff-8d7bed556195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:22.856054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1f308-c325-4916-88ff-8d7bed556195 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:22.857557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1f308-c325-4916-88ff-8d7bed556195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:22.863940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1f308-c325-4916-88ff-8d7bed556195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:22.863940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1f308-c325-4916-88ff-8d7bed556195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:22.863940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1f308-c325-4916-88ff-8d7bed556195 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:22.882807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 887/1792] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:22.884356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9f33e-31b9-4032-b0fb-9a2ae3e7870f None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:22.884776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9f33e-31b9-4032-b0fb-9a2ae3e7870f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:22.885149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9f33e-31b9-4032-b0fb-9a2ae3e7870f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:22.898443 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 906/1793] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:22.898443 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1f308-c325-4916-88ff-8d7bed556195 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:22.912287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1f308-c325-4916-88ff-8d7bed556195 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:22.921622 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a9bdd-490c-49f7-8560-507134d78b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:22.925298 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a9bdd-490c-49f7-8560-507134d78b2b admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:22.925767 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a9bdd-490c-49f7-8560-507134d78b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:22.926365 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a9bdd-490c-49f7-8560-507134d78b2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:22.927243 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a9bdd-490c-49f7-8560-507134d78b2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:22.929881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a9bdd-490c-49f7-8560-507134d78b2b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:22.968001 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-03f9f33e-31b9-4032-b0fb-9a2ae3e7870f None None] MFA Rules not processed for user `b88dd5b396354ff9a4effb30e4ac8572`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:22.975171 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-03f9f33e-31b9-4032-b0fb-9a2ae3e7870f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:23.044828 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 907/1794] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:22 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:23.050162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a9bdd-490c-49f7-8560-507134d78b2b admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:23.181572 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 888/1795] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:23.182918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9f33e-31b9-4032-b0fb-9a2ae3e7870f None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:23.209024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9f33e-31b9-4032-b0fb-9a2ae3e7870f None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:23.211135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a9bdd-490c-49f7-8560-507134d78b2b admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:23.215046 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73390fe1-c3fa-4fc0-a03c-19af112cd45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:23.216284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73390fe1-c3fa-4fc0-a03c-19af112cd45d admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:23.217001 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73390fe1-c3fa-4fc0-a03c-19af112cd45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:23.217421 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73390fe1-c3fa-4fc0-a03c-19af112cd45d admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:23.218292 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73390fe1-c3fa-4fc0-a03c-19af112cd45d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:23.228050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6f90a-b008-4bfb-98db-6a844741ae47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:23.229013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6f90a-b008-4bfb-98db-6a844741ae47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:23.229402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6f90a-b008-4bfb-98db-6a844741ae47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:23.229785 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6f90a-b008-4bfb-98db-6a844741ae47 admin admin] PATH_INFO: `/v3/projects/fe1e0cd3163c42f0b2a42d7a0ebd8d60/users/bb28fbe8b93e4d5686800dd17b283085/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:23.231547 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73390fe1-c3fa-4fc0-a03c-19af112cd45d admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:23.234668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6f90a-b008-4bfb-98db-6a844741ae47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1e0cd3163c42f0b2a42d7a0ebd8d60, user_id=bb28fbe8b93e4d5686800dd17b283085, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:23.240845 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6f90a-b008-4bfb-98db-6a844741ae47 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:23.272911 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 908/1796] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:23 2022] PUT /identity/v3/projects/fe1e0cd3163c42f0b2a42d7a0ebd8d60/users/bb28fbe8b93e4d5686800dd17b283085/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:23.285578 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6f90a-b008-4bfb-98db-6a844741ae47 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:23.338274 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:23.360250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 889/1797] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:23 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:23.362349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023dfb8-397a-46e5-8c2c-5928eff2e156 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:23.363887 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023dfb8-397a-46e5-8c2c-5928eff2e156 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:23.365191 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023dfb8-397a-46e5-8c2c-5928eff2e156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:23.398185 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:23.470547 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-2023dfb8-397a-46e5-8c2c-5928eff2e156 None None] MFA Rules not processed for user `bb28fbe8b93e4d5686800dd17b283085`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:23.481201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2023dfb8-397a-46e5-8c2c-5928eff2e156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:23.514967 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6f90a-b008-4bfb-98db-6a844741ae47 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:23.528669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fc763-c492-4acd-a79c-70099ad9f103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:23.529318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fc763-c492-4acd-a79c-70099ad9f103 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:23.529731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fc763-c492-4acd-a79c-70099ad9f103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:23.529852 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fc763-c492-4acd-a79c-70099ad9f103 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:23.530487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fc763-c492-4acd-a79c-70099ad9f103 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:23.533065 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fc763-c492-4acd-a79c-70099ad9f103 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:23.613315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 890/1798] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:23.614587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023dfb8-397a-46e5-8c2c-5928eff2e156 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:23.631498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023dfb8-397a-46e5-8c2c-5928eff2e156 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:23.639395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a7627-7b08-4785-b6bf-b3056e1b3401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:23.639710 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a7627-7b08-4785-b6bf-b3056e1b3401 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:23.640085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a7627-7b08-4785-b6bf-b3056e1b3401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:23.640429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a7627-7b08-4785-b6bf-b3056e1b3401 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:23.641688 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a7627-7b08-4785-b6bf-b3056e1b3401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:23.643496 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a7627-7b08-4785-b6bf-b3056e1b3401 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:23.669102 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 891/1799] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:23.671092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a7627-7b08-4785-b6bf-b3056e1b3401 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:23.674123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-df6a7627-7b08-4785-b6bf-b3056e1b3401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:23.694749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 909/1800] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:23 2022] POST /identity/v3/projects => generated 375 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:23.694749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fc763-c492-4acd-a79c-70099ad9f103 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:23.701729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e91fc763-c492-4acd-a79c-70099ad9f103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:23.783288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a7627-7b08-4785-b6bf-b3056e1b3401 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:23.876919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64283243-a5c6-4b48-bd71-0483f1005d7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:23.878255 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64283243-a5c6-4b48-bd71-0483f1005d7a service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:23.879991 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fc763-c492-4acd-a79c-70099ad9f103 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:23.880665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64283243-a5c6-4b48-bd71-0483f1005d7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:23.881488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64283243-a5c6-4b48-bd71-0483f1005d7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:23.889415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f653ff1-3c0e-4d92-a196-08673f75fadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:23.889689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f653ff1-3c0e-4d92-a196-08673f75fadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:23.889966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f653ff1-3c0e-4d92-a196-08673f75fadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:23.890237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f653ff1-3c0e-4d92-a196-08673f75fadd admin admin] PATH_INFO: `/v3/projects/0da08fee56ad48c6932daef85f31a9c5` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:23.893374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f653ff1-3c0e-4d92-a196-08673f75fadd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0da08fee56ad48c6932daef85f31a9c5)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:23.905810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f653ff1-3c0e-4d92-a196-08673f75fadd admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:23.978542 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:24.008311 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64283243-a5c6-4b48-bd71-0483f1005d7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:24.011423 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64283243-a5c6-4b48-bd71-0483f1005d7a service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:24.043483 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 892/1801] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:24.044512 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64283243-a5c6-4b48-bd71-0483f1005d7a service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:24.072332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64283243-a5c6-4b48-bd71-0483f1005d7a service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:24.073030 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3a4b1-c041-4fbc-93aa-c9ce6a4ea62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:24.073706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3a4b1-c041-4fbc-93aa-c9ce6a4ea62d admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:24.073978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3a4b1-c041-4fbc-93aa-c9ce6a4ea62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:24.074242 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3a4b1-c041-4fbc-93aa-c9ce6a4ea62d admin admin] PATH_INFO: `/v3/projects/5b7fb8d73d984ce284490d2cf092f479/users/312c439c4881416c8fdc381c7b322178/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:24.098709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3a4b1-c041-4fbc-93aa-c9ce6a4ea62d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7fb8d73d984ce284490d2cf092f479, user_id=312c439c4881416c8fdc381c7b322178, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:24.107584 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3a4b1-c041-4fbc-93aa-c9ce6a4ea62d admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:24.110302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-9f653ff1-3c0e-4d92-a196-08673f75fadd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0da08fee56ad48c6932daef85f31a9c5', 'request_id': 'req-9f653ff1-3c0e-4d92-a196-08673f75fadd'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:24.123741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-9f653ff1-3c0e-4d92-a196-08673f75fadd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0da08fee56ad48c6932daef85f31a9c5', 'request_id': 'req-9f653ff1-3c0e-4d92-a196-08673f75fadd'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:24.131406 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-9f653ff1-3c0e-4d92-a196-08673f75fadd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0da08fee56ad48c6932daef85f31a9c5', 'request_id': 'req-9f653ff1-3c0e-4d92-a196-08673f75fadd'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:24.199321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 893/1802] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:24 2022] PUT /identity/v3/projects/5b7fb8d73d984ce284490d2cf092f479/users/312c439c4881416c8fdc381c7b322178/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:24.199321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3a4b1-c041-4fbc-93aa-c9ce6a4ea62d admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:24.216235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-9f653ff1-3c0e-4d92-a196-08673f75fadd admin admin] The token cache is being invalidate because project 0da08fee56ad48c6932daef85f31a9c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:24.216533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-9f653ff1-3c0e-4d92-a196-08673f75fadd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:24.218594 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-9f653ff1-3c0e-4d92-a196-08673f75fadd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:24.220542 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 910/1803] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:23 2022] DELETE /identity/v3/projects/0da08fee56ad48c6932daef85f31a9c5 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:24.221601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f653ff1-3c0e-4d92-a196-08673f75fadd admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:24.225566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9f653ff1-3c0e-4d92-a196-08673f75fadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:24.339861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3a4b1-c041-4fbc-93aa-c9ce6a4ea62d admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:24.350372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c6981-2323-48e7-906b-1caec8c3ea3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:24.351201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c6981-2323-48e7-906b-1caec8c3ea3f admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:24.351451 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c6981-2323-48e7-906b-1caec8c3ea3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:24.351728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c6981-2323-48e7-906b-1caec8c3ea3f admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:24.352485 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c6981-2323-48e7-906b-1caec8c3ea3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:24.354703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c6981-2323-48e7-906b-1caec8c3ea3f admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:24.366857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:24.393993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f653ff1-3c0e-4d92-a196-08673f75fadd admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:24.412691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc7f6a-3ba5-4611-900e-dca33ff77418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:24.413654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc7f6a-3ba5-4611-900e-dca33ff77418 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:24.414107 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc7f6a-3ba5-4611-900e-dca33ff77418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:24.414650 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc7f6a-3ba5-4611-900e-dca33ff77418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:24.415760 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc7f6a-3ba5-4611-900e-dca33ff77418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:24.417718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc7f6a-3ba5-4611-900e-dca33ff77418 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:24.430838 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 911/1804] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:24.432490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dba6a-28c7-4284-947a-dba60a5e9044 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:24.433007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dba6a-28c7-4284-947a-dba60a5e9044 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:24.433499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dba6a-28c7-4284-947a-dba60a5e9044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:24.490791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 894/1805] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:24 2022] POST /identity/v3/users => generated 294 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:24.490791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c6981-2323-48e7-906b-1caec8c3ea3f admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:24.496069 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-748c6981-2323-48e7-906b-1caec8c3ea3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:24.503225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-464dba6a-28c7-4284-947a-dba60a5e9044 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:24.509763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-464dba6a-28c7-4284-947a-dba60a5e9044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:24.536818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 912/1806] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:24.536818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464dba6a-28c7-4284-947a-dba60a5e9044 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:24.563016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464dba6a-28c7-4284-947a-dba60a5e9044 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:24.584526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2b08f-b436-4971-9834-66634386f392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:24.586004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2b08f-b436-4971-9834-66634386f392 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:24.586672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2b08f-b436-4971-9834-66634386f392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:24.587134 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2b08f-b436-4971-9834-66634386f392 admin admin] PATH_INFO: `/v3/projects/5b7fb8d73d984ce284490d2cf092f479/users/312c439c4881416c8fdc381c7b322178/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:24.590589 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2b08f-b436-4971-9834-66634386f392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7fb8d73d984ce284490d2cf092f479, user_id=312c439c4881416c8fdc381c7b322178, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:24.595703 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2b08f-b436-4971-9834-66634386f392 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:24.605934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:24.624115 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 913/1807] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:24 2022] PUT /identity/v3/projects/5b7fb8d73d984ce284490d2cf092f479/users/312c439c4881416c8fdc381c7b322178/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:24.625345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2b08f-b436-4971-9834-66634386f392 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:24.627280 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-61b2b08f-b436-4971-9834-66634386f392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:24.637946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c6981-2323-48e7-906b-1caec8c3ea3f admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:24.728213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2b08f-b436-4971-9834-66634386f392 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:24.734126 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e6453-a845-46ff-a925-120512ca7246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:24.735153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e6453-a845-46ff-a925-120512ca7246 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:24.735153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e6453-a845-46ff-a925-120512ca7246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:24.739490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e6453-a845-46ff-a925-120512ca7246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:24.739490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e6453-a845-46ff-a925-120512ca7246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:24.739490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e6453-a845-46ff-a925-120512ca7246 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:24.755246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 914/1808] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:24.755246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7109b-b32b-4306-b106-6e71d0ad538a None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:24.755246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7109b-b32b-4306-b106-6e71d0ad538a None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:24.755246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7109b-b32b-4306-b106-6e71d0ad538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:24.755654 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffaa8bd-8612-4623-a02a-1b3dea1ea708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:24.756161 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaa8bd-8612-4623-a02a-1b3dea1ea708 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:24.756356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaa8bd-8612-4623-a02a-1b3dea1ea708 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:24.756668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaa8bd-8612-4623-a02a-1b3dea1ea708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:24.761054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-dffaa8bd-8612-4623-a02a-1b3dea1ea708 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:24.792389 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-3ee7109b-b32b-4306-b106-6e71d0ad538a None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:24.794613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-3ee7109b-b32b-4306-b106-6e71d0ad538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:24.836907 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 915/1809] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:24.838350 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1a4b0-e880-4918-a454-a748fce3e8d4 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:24.838747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1a4b0-e880-4918-a454-a748fce3e8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:24.839182 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1a4b0-e880-4918-a454-a748fce3e8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:24.852838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffaa8bd-8612-4623-a02a-1b3dea1ea708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:24.854307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffaa8bd-8612-4623-a02a-1b3dea1ea708 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:24.872077 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-3ec1a4b0-e880-4918-a454-a748fce3e8d4 None None] MFA Rules not processed for user `312c439c4881416c8fdc381c7b322178`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:24.873526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-3ec1a4b0-e880-4918-a454-a748fce3e8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:24.883804 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 895/1810] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:24.884675 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffaa8bd-8612-4623-a02a-1b3dea1ea708 service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:24.896150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffaa8bd-8612-4623-a02a-1b3dea1ea708 service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:24.901629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c4db-d59f-4dc5-a044-2135a7c82922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:24.902298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c4db-d59f-4dc5-a044-2135a7c82922 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:24.902709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c4db-d59f-4dc5-a044-2135a7c82922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:24.902999 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c4db-d59f-4dc5-a044-2135a7c82922 admin admin] PATH_INFO: `/v3/projects/1cace3c427004de4a29eae1d9dfc2fc1/users/106dddad8d594c3290f436cde1b4e6b8/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:24.927228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c4db-d59f-4dc5-a044-2135a7c82922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cace3c427004de4a29eae1d9dfc2fc1, user_id=106dddad8d594c3290f436cde1b4e6b8, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:24.930486 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421c4db-d59f-4dc5-a044-2135a7c82922 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:24.966507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 916/1811] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:24.978337 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1a4b0-e880-4918-a454-a748fce3e8d4 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:25.009403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1a4b0-e880-4918-a454-a748fce3e8d4 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:25.015984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 896/1812] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:24 2022] PUT /identity/v3/projects/1cace3c427004de4a29eae1d9dfc2fc1/users/106dddad8d594c3290f436cde1b4e6b8/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:25.017686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c4db-d59f-4dc5-a044-2135a7c82922 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:25.022234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d421c4db-d59f-4dc5-a044-2135a7c82922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:25.142609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c4db-d59f-4dc5-a044-2135a7c82922 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:25.148447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba55dfd-ceb5-4ad6-a42c-d7d9cf578b64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:25.149115 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba55dfd-ceb5-4ad6-a42c-d7d9cf578b64 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:25.149399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba55dfd-ceb5-4ad6-a42c-d7d9cf578b64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:25.149675 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba55dfd-ceb5-4ad6-a42c-d7d9cf578b64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:25.191292 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d90cd-0797-48a1-8dd1-629f8950731c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:25.192133 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d90cd-0797-48a1-8dd1-629f8950731c admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:25.192366 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d90cd-0797-48a1-8dd1-629f8950731c admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:25.192663 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d90cd-0797-48a1-8dd1-629f8950731c admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:25.196672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d90cd-0797-48a1-8dd1-629f8950731c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:19:25.196672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d90cd-0797-48a1-8dd1-629f8950731c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:25.199849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d90cd-0797-48a1-8dd1-629f8950731c admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:25.246504 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 917/1813] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:19:25.247236 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d90cd-0797-48a1-8dd1-629f8950731c admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:25.292388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d90cd-0797-48a1-8dd1-629f8950731c admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:25.321319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba55dfd-ceb5-4ad6-a42c-d7d9cf578b64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:25.323560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba55dfd-ceb5-4ad6-a42c-d7d9cf578b64 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:25.326471 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fa11f-3954-4b49-b9d2-5f8e3bf68668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:25.327303 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fa11f-3954-4b49-b9d2-5f8e3bf68668 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:25.328453 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fa11f-3954-4b49-b9d2-5f8e3bf68668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:25.330012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fa11f-3954-4b49-b9d2-5f8e3bf68668 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:25.334190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17fa11f-3954-4b49-b9d2-5f8e3bf68668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:25.334190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17fa11f-3954-4b49-b9d2-5f8e3bf68668 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:25.336293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 897/1814] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:25.337379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba55dfd-ceb5-4ad6-a42c-d7d9cf578b64 service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:25.365529 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba55dfd-ceb5-4ad6-a42c-d7d9cf578b64 service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:25.372269 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea26cf-c47e-44ed-8551-68b73546ae18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:25.372923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea26cf-c47e-44ed-8551-68b73546ae18 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:25.373261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea26cf-c47e-44ed-8551-68b73546ae18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:25.373547 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea26cf-c47e-44ed-8551-68b73546ae18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:25.374266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea26cf-c47e-44ed-8551-68b73546ae18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:25.378958 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea26cf-c47e-44ed-8551-68b73546ae18 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:25.379679 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 918/1815] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:25.383027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fa11f-3954-4b49-b9d2-5f8e3bf68668 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:25.390232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:25.400687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fa11f-3954-4b49-b9d2-5f8e3bf68668 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:25.424716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea8b34-7717-4fe0-9096-16465e5b72fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:25.425494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea8b34-7717-4fe0-9096-16465e5b72fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:25.428776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea8b34-7717-4fe0-9096-16465e5b72fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:25.428776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea8b34-7717-4fe0-9096-16465e5b72fe admin admin] PATH_INFO: `/v3/projects/1cace3c427004de4a29eae1d9dfc2fc1/users/106dddad8d594c3290f436cde1b4e6b8/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:25.429329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea8b34-7717-4fe0-9096-16465e5b72fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cace3c427004de4a29eae1d9dfc2fc1, user_id=106dddad8d594c3290f436cde1b4e6b8, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:25.441200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea8b34-7717-4fe0-9096-16465e5b72fe admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:25.473861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 919/1816] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:25 2022] PUT /identity/v3/projects/1cace3c427004de4a29eae1d9dfc2fc1/users/106dddad8d594c3290f436cde1b4e6b8/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:25.484690 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927844f-27bb-4e5f-b09a-c0d64082198f None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:25.485259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927844f-27bb-4e5f-b09a-c0d64082198f None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:25.485713 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927844f-27bb-4e5f-b09a-c0d64082198f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:25.504243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 898/1817] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:25 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:25.510739 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea26cf-c47e-44ed-8551-68b73546ae18 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:25.574856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-8927844f-27bb-4e5f-b09a-c0d64082198f None None] MFA Rules not processed for user `106dddad8d594c3290f436cde1b4e6b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:25.576362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-8927844f-27bb-4e5f-b09a-c0d64082198f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:25.745152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 920/1818] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:25.784068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea26cf-c47e-44ed-8551-68b73546ae18 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:25.821206 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a48600f-1f34-45e2-ac41-eab065c1c219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:25.833136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48600f-1f34-45e2-ac41-eab065c1c219 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:25.833136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48600f-1f34-45e2-ac41-eab065c1c219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:25.833136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48600f-1f34-45e2-ac41-eab065c1c219 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:25.833136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a48600f-1f34-45e2-ac41-eab065c1c219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:25.833136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a48600f-1f34-45e2-ac41-eab065c1c219 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:25.833493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927844f-27bb-4e5f-b09a-c0d64082198f None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:25.851378 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:25.964354 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 899/1819] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:25 2022] POST /identity/v3/users => generated 294 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:25.965473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a48600f-1f34-45e2-ac41-eab065c1c219 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:26.015237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927844f-27bb-4e5f-b09a-c0d64082198f None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:26.015394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a48600f-1f34-45e2-ac41-eab065c1c219 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:26.024179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e095672-05f9-486d-b5ef-84a7432844b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:26.025237 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e095672-05f9-486d-b5ef-84a7432844b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:26.025761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e095672-05f9-486d-b5ef-84a7432844b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:26.026434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e095672-05f9-486d-b5ef-84a7432844b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:26.026934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc33e5-7bc1-45dd-8171-ce634cb4ec1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:26.027642 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc33e5-7bc1-45dd-8171-ce634cb4ec1b service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:26.027933 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc33e5-7bc1-45dd-8171-ce634cb4ec1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:26.028215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc33e5-7bc1-45dd-8171-ce634cb4ec1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:26.028649 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e095672-05f9-486d-b5ef-84a7432844b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:26.030732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e095672-05f9-486d-b5ef-84a7432844b2 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:26.031486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-3ccc33e5-7bc1-45dd-8171-ce634cb4ec1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:26.050196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 900/1820] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:26.054481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e095672-05f9-486d-b5ef-84a7432844b2 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:26.079768 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e095672-05f9-486d-b5ef-84a7432844b2 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:26.095184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c881f3-aed2-4a5f-943a-7a100273551e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:26.095800 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c881f3-aed2-4a5f-943a-7a100273551e admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:26.096108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c881f3-aed2-4a5f-943a-7a100273551e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:26.096435 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c881f3-aed2-4a5f-943a-7a100273551e admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:26.098995 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c881f3-aed2-4a5f-943a-7a100273551e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:26.104011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c881f3-aed2-4a5f-943a-7a100273551e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:26.167865 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 901/1821] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:26.175050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c881f3-aed2-4a5f-943a-7a100273551e admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:26.196219 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc33e5-7bc1-45dd-8171-ce634cb4ec1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:26.201399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c881f3-aed2-4a5f-943a-7a100273551e admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:26.202176 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc33e5-7bc1-45dd-8171-ce634cb4ec1b service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:26.209595 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67cb60-a78b-4a67-bab5-878617ce9687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:26.227045 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 921/1822] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:26.239770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67cb60-a78b-4a67-bab5-878617ce9687 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:26.239967 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67cb60-a78b-4a67-bab5-878617ce9687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:26.243413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67cb60-a78b-4a67-bab5-878617ce9687 admin admin] PATH_INFO: `/v3/projects/5af9e70d9d7f4a88a3a2ea0bb89fbdab/users/2ea838a0d7254593945666a5f8f018e7/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:26.295708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67cb60-a78b-4a67-bab5-878617ce9687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5af9e70d9d7f4a88a3a2ea0bb89fbdab, user_id=2ea838a0d7254593945666a5f8f018e7, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:26.295708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67cb60-a78b-4a67-bab5-878617ce9687 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:26.398298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 902/1823] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:26 2022] PUT /identity/v3/projects/5af9e70d9d7f4a88a3a2ea0bb89fbdab/users/2ea838a0d7254593945666a5f8f018e7/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:26.412448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0cbff-c506-49a2-8caa-4db960e62ae2 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:26.416235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0cbff-c506-49a2-8caa-4db960e62ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:26.416235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0cbff-c506-49a2-8caa-4db960e62ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:26.496218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-bec0cbff-c506-49a2-8caa-4db960e62ae2 None None] MFA Rules not processed for user `2ea838a0d7254593945666a5f8f018e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:26.497598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-bec0cbff-c506-49a2-8caa-4db960e62ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:26.612430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 922/1824] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:26.617588 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67cb60-a78b-4a67-bab5-878617ce9687 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:26.722762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67cb60-a78b-4a67-bab5-878617ce9687 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:26.728942 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8676d5-500d-422c-ab1a-f2515deea0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:26.729574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8676d5-500d-422c-ab1a-f2515deea0cf admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:26.729860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8676d5-500d-422c-ab1a-f2515deea0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:26.730140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8676d5-500d-422c-ab1a-f2515deea0cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:26.730852 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8676d5-500d-422c-ab1a-f2515deea0cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:26.732786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8676d5-500d-422c-ab1a-f2515deea0cf admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:26.838710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0cbff-c506-49a2-8caa-4db960e62ae2 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:26.849379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 903/1825] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:26 2022] POST /identity/v3/projects => generated 387 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:26.851321 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0cbff-c506-49a2-8caa-4db960e62ae2 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:26.958650 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699202d-2308-48a6-ba60-5c0f5b0558bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:26.960066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699202d-2308-48a6-ba60-5c0f5b0558bf admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:26.960066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699202d-2308-48a6-ba60-5c0f5b0558bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:26.960066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699202d-2308-48a6-ba60-5c0f5b0558bf admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:26.961279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699202d-2308-48a6-ba60-5c0f5b0558bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:26.963297 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699202d-2308-48a6-ba60-5c0f5b0558bf admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:27.116795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 923/1826] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:26 2022] POST /identity/v3/users => generated 300 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:19:27.123073 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8676d5-500d-422c-ab1a-f2515deea0cf admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:27.137523 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8676d5-500d-422c-ab1a-f2515deea0cf admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:27.143999 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bd252-371c-4409-ab14-ceabeda41960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:27.144627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bd252-371c-4409-ab14-ceabeda41960 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:27.144966 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bd252-371c-4409-ab14-ceabeda41960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:27.147602 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bd252-371c-4409-ab14-ceabeda41960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:27.148305 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205bd252-371c-4409-ab14-ceabeda41960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:27.150485 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205bd252-371c-4409-ab14-ceabeda41960 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:27.163568 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 904/1827] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:27.168324 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699202d-2308-48a6-ba60-5c0f5b0558bf admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:27.197578 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699202d-2308-48a6-ba60-5c0f5b0558bf admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:27.209816 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67cb2d-0683-489c-b188-3f483e96ebaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:27.210508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67cb2d-0683-489c-b188-3f483e96ebaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:27.210834 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67cb2d-0683-489c-b188-3f483e96ebaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:27.211161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67cb2d-0683-489c-b188-3f483e96ebaf admin admin] PATH_INFO: `/v3/projects/c4720f8189bd4b8ebf0afc3fc9c2fd2c/users/6f21d2ae1caa468d8afa0a8391374d07/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:27.251594 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67cb2d-0683-489c-b188-3f483e96ebaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4720f8189bd4b8ebf0afc3fc9c2fd2c, user_id=6f21d2ae1caa468d8afa0a8391374d07, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:27.253447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:27.259563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67cb2d-0683-489c-b188-3f483e96ebaf admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:27.304271 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bd252-371c-4409-ab14-ceabeda41960 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:27.392070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bd252-371c-4409-ab14-ceabeda41960 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:27.399706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a86f9-a231-4b14-b946-9776784461ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:27.400965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a86f9-a231-4b14-b946-9776784461ff service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:27.401418 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a86f9-a231-4b14-b946-9776784461ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:27.401866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a86f9-a231-4b14-b946-9776784461ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:27.423797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c18a86f9-a231-4b14-b946-9776784461ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:27.507562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a86f9-a231-4b14-b946-9776784461ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:27.509488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a86f9-a231-4b14-b946-9776784461ff service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:27.528764 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 905/1828] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:27.581574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 924/1829] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:27 2022] PUT /identity/v3/projects/c4720f8189bd4b8ebf0afc3fc9c2fd2c/users/6f21d2ae1caa468d8afa0a8391374d07/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:27.585627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a86f9-a231-4b14-b946-9776784461ff service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:27.602002 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a86f9-a231-4b14-b946-9776784461ff service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:27.680695 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd87c3-8ff2-4987-80d5-40e10c80dab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:27.681304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd87c3-8ff2-4987-80d5-40e10c80dab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:27.681585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd87c3-8ff2-4987-80d5-40e10c80dab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:27.681856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd87c3-8ff2-4987-80d5-40e10c80dab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:27.682556 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd87c3-8ff2-4987-80d5-40e10c80dab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:27.684180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd87c3-8ff2-4987-80d5-40e10c80dab6 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:27.694665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 906/1830] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:27.699351 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67cb2d-0683-489c-b188-3f483e96ebaf admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:27.710271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67cb2d-0683-489c-b188-3f483e96ebaf admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:27.714872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e7617-068d-4744-b33e-61c0b6cad19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:27.715514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e7617-068d-4744-b33e-61c0b6cad19d admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:27.715794 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e7617-068d-4744-b33e-61c0b6cad19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:27.716066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e7617-068d-4744-b33e-61c0b6cad19d admin admin] PATH_INFO: `/v3/projects/c4720f8189bd4b8ebf0afc3fc9c2fd2c/users/6f21d2ae1caa468d8afa0a8391374d07/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:27.718001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e7617-068d-4744-b33e-61c0b6cad19d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4720f8189bd4b8ebf0afc3fc9c2fd2c, user_id=6f21d2ae1caa468d8afa0a8391374d07, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:27.720699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e7617-068d-4744-b33e-61c0b6cad19d admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:27.733458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 925/1831] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:27 2022] PUT /identity/v3/projects/c4720f8189bd4b8ebf0afc3fc9c2fd2c/users/6f21d2ae1caa468d8afa0a8391374d07/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:27.744179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af398483-1480-4dd5-9886-e4eede1d631e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:27.744472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af398483-1480-4dd5-9886-e4eede1d631e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:27.744777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af398483-1480-4dd5-9886-e4eede1d631e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:27.774335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-af398483-1480-4dd5-9886-e4eede1d631e None None] MFA Rules not processed for user `6f21d2ae1caa468d8afa0a8391374d07`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:27.775590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-af398483-1480-4dd5-9886-e4eede1d631e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:27.861020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 907/1832] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:27.873089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e7617-068d-4744-b33e-61c0b6cad19d admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:27.930001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e7617-068d-4744-b33e-61c0b6cad19d admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:27.933614 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6c658-ca96-4495-ae43-0323cd66c18c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:27.934206 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6c658-ca96-4495-ae43-0323cd66c18c service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:27.934485 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6c658-ca96-4495-ae43-0323cd66c18c service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:27.934756 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6c658-ca96-4495-ae43-0323cd66c18c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:27.944313 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6c658-ca96-4495-ae43-0323cd66c18c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:27.945746 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6c658-ca96-4495-ae43-0323cd66c18c service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:27.954907 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 926/1833] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:28.761757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:28.896855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:29.227489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af398483-1480-4dd5-9886-e4eede1d631e None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:29.294504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af398483-1480-4dd5-9886-e4eede1d631e None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:29.297701 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33da73-3c93-42ae-802c-08df73ddd9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:29.298310 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33da73-3c93-42ae-802c-08df73ddd9e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:29.298601 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33da73-3c93-42ae-802c-08df73ddd9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:29.298928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33da73-3c93-42ae-802c-08df73ddd9e4 admin admin] PATH_INFO: `/v3/users/2ea838a0d7254593945666a5f8f018e7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:29.300261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33da73-3c93-42ae-802c-08df73ddd9e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ea838a0d7254593945666a5f8f018e7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:29.302251 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33da73-3c93-42ae-802c-08df73ddd9e4 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:29.341817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:29.534797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2e33da73-3c93-42ae-802c-08df73ddd9e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ea838a0d7254593945666a5f8f018e7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:29.535466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2e33da73-3c93-42ae-802c-08df73ddd9e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ea838a0d7254593945666a5f8f018e7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:29.670430 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2e33da73-3c93-42ae-802c-08df73ddd9e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ea838a0d7254593945666a5f8f018e7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:29.684564 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2e33da73-3c93-42ae-802c-08df73ddd9e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ea838a0d7254593945666a5f8f018e7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:29.692192 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2e33da73-3c93-42ae-802c-08df73ddd9e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ea838a0d7254593945666a5f8f018e7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:29.698346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 908/1834] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:29 2022] DELETE /identity/v3/users/2ea838a0d7254593945666a5f8f018e7 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:29.702969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3728a2-5620-4ca8-a6fe-a599f27b9dce None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:29.703259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3728a2-5620-4ca8-a6fe-a599f27b9dce None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:29.703538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3728a2-5620-4ca8-a6fe-a599f27b9dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:29.723004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-6a3728a2-5620-4ca8-a6fe-a599f27b9dce None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:29.725842 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-6a3728a2-5620-4ca8-a6fe-a599f27b9dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:29.790765 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 927/1835] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:29.801561 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33da73-3c93-42ae-802c-08df73ddd9e4 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:29.802988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2e33da73-3c93-42ae-802c-08df73ddd9e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:29.875953 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33da73-3c93-42ae-802c-08df73ddd9e4 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:29.879642 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efaefdb-98c7-42fa-84f1-27fab75e0f71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:29.880256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efaefdb-98c7-42fa-84f1-27fab75e0f71 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:29.880534 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efaefdb-98c7-42fa-84f1-27fab75e0f71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:29.880878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efaefdb-98c7-42fa-84f1-27fab75e0f71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:29.891427 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efaefdb-98c7-42fa-84f1-27fab75e0f71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:29.892966 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efaefdb-98c7-42fa-84f1-27fab75e0f71 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:29.903402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 909/1836] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:29.951081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3728a2-5620-4ca8-a6fe-a599f27b9dce None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:29.951777 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-6a3728a2-5620-4ca8-a6fe-a599f27b9dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:29.966002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3728a2-5620-4ca8-a6fe-a599f27b9dce None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:29.972574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe831b8-ad22-4e0e-bdb4-3fadbe181151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:29.974061 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe831b8-ad22-4e0e-bdb4-3fadbe181151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:29.974513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe831b8-ad22-4e0e-bdb4-3fadbe181151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:29.974951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe831b8-ad22-4e0e-bdb4-3fadbe181151 admin admin] PATH_INFO: `/v3/users/25cb9992166e4464a72fd0741edd9bb6` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:29.976361 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe831b8-ad22-4e0e-bdb4-3fadbe181151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25cb9992166e4464a72fd0741edd9bb6)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:29.979020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe831b8-ad22-4e0e-bdb4-3fadbe181151 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:30.193656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:30.257557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-6fe831b8-ad22-4e0e-bdb4-3fadbe181151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25cb9992166e4464a72fd0741edd9bb6'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:30.350047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-6fe831b8-ad22-4e0e-bdb4-3fadbe181151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25cb9992166e4464a72fd0741edd9bb6'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:30.350783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-6fe831b8-ad22-4e0e-bdb4-3fadbe181151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25cb9992166e4464a72fd0741edd9bb6'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:30.358040 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-6fe831b8-ad22-4e0e-bdb4-3fadbe181151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25cb9992166e4464a72fd0741edd9bb6'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:30.371005 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-6fe831b8-ad22-4e0e-bdb4-3fadbe181151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25cb9992166e4464a72fd0741edd9bb6'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:30.384126 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 928/1837] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:29 2022] DELETE /identity/v3/users/25cb9992166e4464a72fd0741edd9bb6 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:30.385291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efaefdb-98c7-42fa-84f1-27fab75e0f71 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:30.385842 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be1421-41f7-48c7-a5e9-60fd21f5f884 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:30.389670 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be1421-41f7-48c7-a5e9-60fd21f5f884 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:30.389670 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be1421-41f7-48c7-a5e9-60fd21f5f884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:30.389863 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1efaefdb-98c7-42fa-84f1-27fab75e0f71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:30.420025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-b5be1421-41f7-48c7-a5e9-60fd21f5f884 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:30.421418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b5be1421-41f7-48c7-a5e9-60fd21f5f884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:30.427768 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efaefdb-98c7-42fa-84f1-27fab75e0f71 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:30.501216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 929/1838] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:30.513776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:30.513776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:30.513776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:30.513776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e admin admin] PATH_INFO: `/v3/projects/5af9e70d9d7f4a88a3a2ea0bb89fbdab` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:30.514123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5af9e70d9d7f4a88a3a2ea0bb89fbdab)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:30.514212 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:30.514300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:30.514386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be1421-41f7-48c7-a5e9-60fd21f5f884 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:30.518875 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b5be1421-41f7-48c7-a5e9-60fd21f5f884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:30.576817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be1421-41f7-48c7-a5e9-60fd21f5f884 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:30.580369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d07e0-7790-4d27-8ed0-4000ddcae469 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:30.580995 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d07e0-7790-4d27-8ed0-4000ddcae469 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:30.581277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d07e0-7790-4d27-8ed0-4000ddcae469 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:30.581549 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d07e0-7790-4d27-8ed0-4000ddcae469 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:30.590201 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d07e0-7790-4d27-8ed0-4000ddcae469 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:30.591614 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d07e0-7790-4d27-8ed0-4000ddcae469 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:30.601906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 930/1839] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:19:30.618026 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5af9e70d9d7f4a88a3a2ea0bb89fbdab', 'request_id': 'req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:30.623449 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5af9e70d9d7f4a88a3a2ea0bb89fbdab', 'request_id': 'req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:30.630612 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5af9e70d9d7f4a88a3a2ea0bb89fbdab', 'request_id': 'req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:30.744712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e admin admin] The token cache is being invalidate because project 5af9e70d9d7f4a88a3a2ea0bb89fbdab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:30.748463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:30.748463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:30.748463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 910/1840] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:30 2022] DELETE /identity/v3/projects/5af9e70d9d7f4a88a3a2ea0bb89fbdab => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:30.751851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d07e0-7790-4d27-8ed0-4000ddcae469 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:30.753310 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-087d07e0-7790-4d27-8ed0-4000ddcae469 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:30.766296 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d07e0-7790-4d27-8ed0-4000ddcae469 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:30.835937 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d3d7a-13c2-478f-87b1-de5401842fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:30.836515 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:30.836597 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d3d7a-13c2-478f-87b1-de5401842fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:30.836769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d3d7a-13c2-478f-87b1-de5401842fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:30.837052 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d3d7a-13c2-478f-87b1-de5401842fc0 admin admin] PATH_INFO: `/v3/users/6f21d2ae1caa468d8afa0a8391374d07` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:30.838302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d3d7a-13c2-478f-87b1-de5401842fc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f21d2ae1caa468d8afa0a8391374d07)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:30.840388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d3d7a-13c2-478f-87b1-de5401842fc0 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:31.061793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c48d3d7a-13c2-478f-87b1-de5401842fc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f21d2ae1caa468d8afa0a8391374d07'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.073087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:31.131385 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:31.132839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:31.146853 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6e1ae-d7c6-4557-be4e-903421f5de4e admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:31.149771 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c48d3d7a-13c2-478f-87b1-de5401842fc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f21d2ae1caa468d8afa0a8391374d07'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.152909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c48d3d7a-13c2-478f-87b1-de5401842fc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f21d2ae1caa468d8afa0a8391374d07'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.154960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771daad6-f384-4c44-9c21-543b669e9c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:31.155791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771daad6-f384-4c44-9c21-543b669e9c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:31.156728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771daad6-f384-4c44-9c21-543b669e9c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:31.156728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771daad6-f384-4c44-9c21-543b669e9c30 admin admin] PATH_INFO: `/v3/projects/36f63c08eab1497f80d314e496bc5be9` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:31.159423 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771daad6-f384-4c44-9c21-543b669e9c30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36f63c08eab1497f80d314e496bc5be9)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:31.159514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c48d3d7a-13c2-478f-87b1-de5401842fc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f21d2ae1caa468d8afa0a8391374d07'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.160848 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771daad6-f384-4c44-9c21-543b669e9c30 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:31.173026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c48d3d7a-13c2-478f-87b1-de5401842fc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f21d2ae1caa468d8afa0a8391374d07'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.179962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 931/1841] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:30 2022] DELETE /identity/v3/users/6f21d2ae1caa468d8afa0a8391374d07 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:31.296966 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-771daad6-f384-4c44-9c21-543b669e9c30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36f63c08eab1497f80d314e496bc5be9', 'request_id': 'req-771daad6-f384-4c44-9c21-543b669e9c30'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.303061 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-771daad6-f384-4c44-9c21-543b669e9c30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36f63c08eab1497f80d314e496bc5be9', 'request_id': 'req-771daad6-f384-4c44-9c21-543b669e9c30'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.309510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-771daad6-f384-4c44-9c21-543b669e9c30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36f63c08eab1497f80d314e496bc5be9', 'request_id': 'req-771daad6-f384-4c44-9c21-543b669e9c30'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.386622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-771daad6-f384-4c44-9c21-543b669e9c30 admin admin] The token cache is being invalidate because project 36f63c08eab1497f80d314e496bc5be9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:31.386622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-771daad6-f384-4c44-9c21-543b669e9c30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.386622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-771daad6-f384-4c44-9c21-543b669e9c30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.386622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 911/1842] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:31 2022] DELETE /identity/v3/projects/36f63c08eab1497f80d314e496bc5be9 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:19:31.389418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d3d7a-13c2-478f-87b1-de5401842fc0 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:31.391047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-c48d3d7a-13c2-478f-87b1-de5401842fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:31.457751 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d3d7a-13c2-478f-87b1-de5401842fc0 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:31.492816 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:31.570166 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fbc45-75b2-4acc-b4c6-7144089d5586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:31.570800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fbc45-75b2-4acc-b4c6-7144089d5586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:31.571084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fbc45-75b2-4acc-b4c6-7144089d5586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:31.571362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fbc45-75b2-4acc-b4c6-7144089d5586 admin admin] PATH_INFO: `/v3/users/bb28fbe8b93e4d5686800dd17b283085` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:31.572599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fbc45-75b2-4acc-b4c6-7144089d5586 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb28fbe8b93e4d5686800dd17b283085)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:31.574563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fbc45-75b2-4acc-b4c6-7144089d5586 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:31.700103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771daad6-f384-4c44-9c21-543b669e9c30 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:31.701532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-771daad6-f384-4c44-9c21-543b669e9c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:31.715228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771daad6-f384-4c44-9c21-543b669e9c30 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:31.721180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58ae48-2cc3-4d10-b34b-bbcdcf660e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:31.721840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58ae48-2cc3-4d10-b34b-bbcdcf660e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:31.722207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58ae48-2cc3-4d10-b34b-bbcdcf660e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:31.722375 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58ae48-2cc3-4d10-b34b-bbcdcf660e33 admin admin] PATH_INFO: `/v3/projects/c4720f8189bd4b8ebf0afc3fc9c2fd2c` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:31.723592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58ae48-2cc3-4d10-b34b-bbcdcf660e33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4720f8189bd4b8ebf0afc3fc9c2fd2c)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:31.725550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58ae48-2cc3-4d10-b34b-bbcdcf660e33 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:31.777717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-630fbc45-75b2-4acc-b4c6-7144089d5586 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb28fbe8b93e4d5686800dd17b283085'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.847138 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-630fbc45-75b2-4acc-b4c6-7144089d5586 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb28fbe8b93e4d5686800dd17b283085'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.848122 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-630fbc45-75b2-4acc-b4c6-7144089d5586 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb28fbe8b93e4d5686800dd17b283085'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.852192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:31.853850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ad58ae48-2cc3-4d10-b34b-bbcdcf660e33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4720f8189bd4b8ebf0afc3fc9c2fd2c', 'request_id': 'req-ad58ae48-2cc3-4d10-b34b-bbcdcf660e33'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.857863 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-630fbc45-75b2-4acc-b4c6-7144089d5586 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb28fbe8b93e4d5686800dd17b283085'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.859532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ad58ae48-2cc3-4d10-b34b-bbcdcf660e33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4720f8189bd4b8ebf0afc3fc9c2fd2c', 'request_id': 'req-ad58ae48-2cc3-4d10-b34b-bbcdcf660e33'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.866904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ad58ae48-2cc3-4d10-b34b-bbcdcf660e33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4720f8189bd4b8ebf0afc3fc9c2fd2c', 'request_id': 'req-ad58ae48-2cc3-4d10-b34b-bbcdcf660e33'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.870128 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-630fbc45-75b2-4acc-b4c6-7144089d5586 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb28fbe8b93e4d5686800dd17b283085'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.876315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 932/1843] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:31 2022] DELETE /identity/v3/users/bb28fbe8b93e4d5686800dd17b283085 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:31.960422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ad58ae48-2cc3-4d10-b34b-bbcdcf660e33 admin admin] The token cache is being invalidate because project c4720f8189bd4b8ebf0afc3fc9c2fd2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:31.960656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ad58ae48-2cc3-4d10-b34b-bbcdcf660e33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.961295 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ad58ae48-2cc3-4d10-b34b-bbcdcf660e33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:31.962835 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 912/1844] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:31 2022] DELETE /identity/v3/projects/c4720f8189bd4b8ebf0afc3fc9c2fd2c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:31.968133 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b9aca-712c-49f5-a106-b5c3359d016e None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:31.968488 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b9aca-712c-49f5-a106-b5c3359d016e None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:31.968857 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b9aca-712c-49f5-a106-b5c3359d016e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:31.989381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9d9b9aca-712c-49f5-a106-b5c3359d016e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:31.990573 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9d9b9aca-712c-49f5-a106-b5c3359d016e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:32.047498 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 933/1845] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:32.056208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f99d8-a1e3-44df-9839-c28d547f3921 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:32.056504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f99d8-a1e3-44df-9839-c28d547f3921 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:32.056821 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f99d8-a1e3-44df-9839-c28d547f3921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:32.078410 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-a47f99d8-a1e3-44df-9839-c28d547f3921 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:32.079886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a47f99d8-a1e3-44df-9839-c28d547f3921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:32.091116 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 913/1846] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:32.094956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b9aca-712c-49f5-a106-b5c3359d016e None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:32.107783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b9aca-712c-49f5-a106-b5c3359d016e None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:32.114517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8873a-1457-4a90-a0e3-46dd106b6981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:32.119704 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8873a-1457-4a90-a0e3-46dd106b6981 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:32.119704 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8873a-1457-4a90-a0e3-46dd106b6981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:32.119704 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8873a-1457-4a90-a0e3-46dd106b6981 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:32.119704 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8873a-1457-4a90-a0e3-46dd106b6981 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:19:32.119704 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8873a-1457-4a90-a0e3-46dd106b6981 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:32.119704 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8873a-1457-4a90-a0e3-46dd106b6981 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:32.132668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 934/1847] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:19:32.135894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f99d8-a1e3-44df-9839-c28d547f3921 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:32.143596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:32.154275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f99d8-a1e3-44df-9839-c28d547f3921 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:32.155026 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b492e-361b-4499-b1ce-8b0a36955e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:32.155808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b492e-361b-4499-b1ce-8b0a36955e77 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:32.156097 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b492e-361b-4499-b1ce-8b0a36955e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:32.156354 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b492e-361b-4499-b1ce-8b0a36955e77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:32.157022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b492e-361b-4499-b1ce-8b0a36955e77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:32.159473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b492e-361b-4499-b1ce-8b0a36955e77 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:32.238827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 914/1848] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:32 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:32.239993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8873a-1457-4a90-a0e3-46dd106b6981 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:32.250227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8873a-1457-4a90-a0e3-46dd106b6981 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:32.308826 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023e985-92ca-4917-83fb-ec5ebf5c3546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:32.309443 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023e985-92ca-4917-83fb-ec5ebf5c3546 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:32.309726 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023e985-92ca-4917-83fb-ec5ebf5c3546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:32.310000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023e985-92ca-4917-83fb-ec5ebf5c3546 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:32.310669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023e985-92ca-4917-83fb-ec5ebf5c3546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:32.312614 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023e985-92ca-4917-83fb-ec5ebf5c3546 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:32.399303 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 935/1849] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:32 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:19:32.399460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b492e-361b-4499-b1ce-8b0a36955e77 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:32.399460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b79b492e-361b-4499-b1ce-8b0a36955e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:32.400461 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023e985-92ca-4917-83fb-ec5ebf5c3546 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:32.411574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023e985-92ca-4917-83fb-ec5ebf5c3546 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:32.414433 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:32.418687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2206f-3410-4fee-b52c-a679501a911b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:32.419335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2206f-3410-4fee-b52c-a679501a911b admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:32.426101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2206f-3410-4fee-b52c-a679501a911b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:32.426101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2206f-3410-4fee-b52c-a679501a911b admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:32.426101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2206f-3410-4fee-b52c-a679501a911b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:32.426101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2206f-3410-4fee-b52c-a679501a911b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:32.427546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b492e-361b-4499-b1ce-8b0a36955e77 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:32.432331 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04114a-50a0-455e-bfe1-5d40e8968e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:32.433067 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04114a-50a0-455e-bfe1-5d40e8968e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:32.433279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04114a-50a0-455e-bfe1-5d40e8968e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:32.433551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04114a-50a0-455e-bfe1-5d40e8968e66 admin admin] PATH_INFO: `/v3/projects/fe1e0cd3163c42f0b2a42d7a0ebd8d60` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:32.434010 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 936/1850] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:32.434723 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04114a-50a0-455e-bfe1-5d40e8968e66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe1e0cd3163c42f0b2a42d7a0ebd8d60)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:32.436718 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04114a-50a0-455e-bfe1-5d40e8968e66 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:32.455356 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2206f-3410-4fee-b52c-a679501a911b admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:32.514576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2206f-3410-4fee-b52c-a679501a911b admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:32.514576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798cf79-cf90-42a5-beb8-eba8c3869988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:32.514576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798cf79-cf90-42a5-beb8-eba8c3869988 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:32.514576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798cf79-cf90-42a5-beb8-eba8c3869988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:32.515517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798cf79-cf90-42a5-beb8-eba8c3869988 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:32.515517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798cf79-cf90-42a5-beb8-eba8c3869988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:32.515517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798cf79-cf90-42a5-beb8-eba8c3869988 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:32.515517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 937/1851] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:19:32.518837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798cf79-cf90-42a5-beb8-eba8c3869988 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:32.529312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798cf79-cf90-42a5-beb8-eba8c3869988 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:32.537776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41678f2d-3780-4914-aa8c-7ecd81fc26df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:32.538453 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41678f2d-3780-4914-aa8c-7ecd81fc26df admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:32.538735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41678f2d-3780-4914-aa8c-7ecd81fc26df admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:32.539039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41678f2d-3780-4914-aa8c-7ecd81fc26df admin admin] PATH_INFO: `/v3/projects/9c4e7545107a456fb285f0918b0fd30f/users/13a18524bfe5410bb20cb1a807af87e3/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:32.556432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41678f2d-3780-4914-aa8c-7ecd81fc26df admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4e7545107a456fb285f0918b0fd30f, user_id=13a18524bfe5410bb20cb1a807af87e3, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:32.559744 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41678f2d-3780-4914-aa8c-7ecd81fc26df admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:33.533529 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 938/1852] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:32 2022] PUT /identity/v3/projects/9c4e7545107a456fb285f0918b0fd30f/users/13a18524bfe5410bb20cb1a807af87e3/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 1016 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:33.542839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6a04114a-50a0-455e-bfe1-5d40e8968e66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe1e0cd3163c42f0b2a42d7a0ebd8d60', 'request_id': 'req-6a04114a-50a0-455e-bfe1-5d40e8968e66'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:33.545156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66febb06-a887-40ff-a544-be92d7e97a76 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:33.545442 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66febb06-a887-40ff-a544-be92d7e97a76 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:33.545716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66febb06-a887-40ff-a544-be92d7e97a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:33.548810 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6a04114a-50a0-455e-bfe1-5d40e8968e66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe1e0cd3163c42f0b2a42d7a0ebd8d60', 'request_id': 'req-6a04114a-50a0-455e-bfe1-5d40e8968e66'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:33.555996 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6a04114a-50a0-455e-bfe1-5d40e8968e66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe1e0cd3163c42f0b2a42d7a0ebd8d60', 'request_id': 'req-6a04114a-50a0-455e-bfe1-5d40e8968e66'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:33.572954 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-66febb06-a887-40ff-a544-be92d7e97a76 None None] MFA Rules not processed for user `13a18524bfe5410bb20cb1a807af87e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:33.574153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-66febb06-a887-40ff-a544-be92d7e97a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:33.638087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 939/1853] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:33.640378 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6a04114a-50a0-455e-bfe1-5d40e8968e66 admin admin] The token cache is being invalidate because project fe1e0cd3163c42f0b2a42d7a0ebd8d60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:33.640709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6a04114a-50a0-455e-bfe1-5d40e8968e66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:33.641365 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6a04114a-50a0-455e-bfe1-5d40e8968e66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:33.643767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 915/1854] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:32 2022] DELETE /identity/v3/projects/fe1e0cd3163c42f0b2a42d7a0ebd8d60 => generated 0 bytes in 1247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:19:33.648840 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685130e-56ab-4211-81d0-9ee71d8f9c83 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:33.649185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685130e-56ab-4211-81d0-9ee71d8f9c83 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:33.649405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685130e-56ab-4211-81d0-9ee71d8f9c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:33.655325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04114a-50a0-455e-bfe1-5d40e8968e66 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:33.656631 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6a04114a-50a0-455e-bfe1-5d40e8968e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:33.669300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-d685130e-56ab-4211-81d0-9ee71d8f9c83 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:33.670473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-d685130e-56ab-4211-81d0-9ee71d8f9c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:33.698978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:33.716134 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04114a-50a0-455e-bfe1-5d40e8968e66 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:33.719868 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bf579-98ac-4df0-8862-f46d374f1e2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:33.720548 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bf579-98ac-4df0-8862-f46d374f1e2c service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:33.720867 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bf579-98ac-4df0-8862-f46d374f1e2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:33.721141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bf579-98ac-4df0-8862-f46d374f1e2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:33.722918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0f9bf579-98ac-4df0-8862-f46d374f1e2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:33.731081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 940/1855] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:33.736840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bf579-98ac-4df0-8862-f46d374f1e2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:33.738425 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9bf579-98ac-4df0-8862-f46d374f1e2c service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:33.739063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef41037-fdc5-4601-9729-8146676de24d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:33.739421 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef41037-fdc5-4601-9729-8146676de24d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:33.739705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef41037-fdc5-4601-9729-8146676de24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:33.748115 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 916/1856] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:19:33.760437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-4ef41037-fdc5-4601-9729-8146676de24d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:33.761714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4ef41037-fdc5-4601-9729-8146676de24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:33.770923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 941/1857] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:33.771045 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bf579-98ac-4df0-8862-f46d374f1e2c service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:33.772248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0f9bf579-98ac-4df0-8862-f46d374f1e2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:33.775108 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef41037-fdc5-4601-9729-8146676de24d None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:33.785335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef41037-fdc5-4601-9729-8146676de24d None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:33.785977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bf579-98ac-4df0-8862-f46d374f1e2c service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:33.790628 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69710c62-1549-437a-b7f4-ca50cc3c37e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:33.791075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dc938-5f17-4944-b917-d0881ae3484b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:33.791409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69710c62-1549-437a-b7f4-ca50cc3c37e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:33.791697 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69710c62-1549-437a-b7f4-ca50cc3c37e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:33.791784 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dc938-5f17-4944-b917-d0881ae3484b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:33.791972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69710c62-1549-437a-b7f4-ca50cc3c37e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:33.792058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dc938-5f17-4944-b917-d0881ae3484b admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:33.792253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dc938-5f17-4944-b917-d0881ae3484b admin admin] PATH_INFO: `/v3/users/13a18524bfe5410bb20cb1a807af87e3` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:33.792633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69710c62-1549-437a-b7f4-ca50cc3c37e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:19:33.792962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69710c62-1549-437a-b7f4-ca50cc3c37e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:33.793468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76dc938-5f17-4944-b917-d0881ae3484b admin admin] RBAC: Authorizing `identity:delete_user(user_id=13a18524bfe5410bb20cb1a807af87e3)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:33.794597 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69710c62-1549-437a-b7f4-ca50cc3c37e0 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:33.795388 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76dc938-5f17-4944-b917-d0881ae3484b admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:33.810397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 942/1858] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:19:33.827660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69710c62-1549-437a-b7f4-ca50cc3c37e0 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:33.852524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69710c62-1549-437a-b7f4-ca50cc3c37e0 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:33.859880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ec37e-100b-4832-8119-7f7d65cc80a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:33.860686 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ec37e-100b-4832-8119-7f7d65cc80a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:33.861664 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ec37e-100b-4832-8119-7f7d65cc80a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:33.861817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ec37e-100b-4832-8119-7f7d65cc80a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:33.862697 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ec37e-100b-4832-8119-7f7d65cc80a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:33.864774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ec37e-100b-4832-8119-7f7d65cc80a4 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:33.961261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 943/1859] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:33 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:33.965450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ec37e-100b-4832-8119-7f7d65cc80a4 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:33.990919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c76dc938-5f17-4944-b917-d0881ae3484b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13a18524bfe5410bb20cb1a807af87e3'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:33.992141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c76dc938-5f17-4944-b917-d0881ae3484b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13a18524bfe5410bb20cb1a807af87e3'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:34.031010 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ec37e-100b-4832-8119-7f7d65cc80a4 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:34.034577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:34.039409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780414b7-9689-4fdb-91f7-1082a5e8083c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:34.040339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780414b7-9689-4fdb-91f7-1082a5e8083c admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:34.040722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780414b7-9689-4fdb-91f7-1082a5e8083c admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:34.041007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780414b7-9689-4fdb-91f7-1082a5e8083c admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:34.042106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780414b7-9689-4fdb-91f7-1082a5e8083c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:34.043632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780414b7-9689-4fdb-91f7-1082a5e8083c admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:34.057967 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c76dc938-5f17-4944-b917-d0881ae3484b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13a18524bfe5410bb20cb1a807af87e3'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:34.076088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c76dc938-5f17-4944-b917-d0881ae3484b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13a18524bfe5410bb20cb1a807af87e3'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:34.085061 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c76dc938-5f17-4944-b917-d0881ae3484b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13a18524bfe5410bb20cb1a807af87e3'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:34.091834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 917/1860] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:33 2022] DELETE /identity/v3/users/13a18524bfe5410bb20cb1a807af87e3 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:19:34.096931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63598403-9870-46a9-a008-878975c7a950 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:34.097217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63598403-9870-46a9-a008-878975c7a950 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:34.097494 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63598403-9870-46a9-a008-878975c7a950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:34.116565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-63598403-9870-46a9-a008-878975c7a950 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:34.117803 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-63598403-9870-46a9-a008-878975c7a950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:34.159462 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 944/1861] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:33 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:34.160463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780414b7-9689-4fdb-91f7-1082a5e8083c admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:34.162190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-780414b7-9689-4fdb-91f7-1082a5e8083c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:34.174635 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780414b7-9689-4fdb-91f7-1082a5e8083c admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:34.192374 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 918/1862] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:34.203422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63598403-9870-46a9-a008-878975c7a950 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:34.205551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-63598403-9870-46a9-a008-878975c7a950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:34.242892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b6b9e-03fa-49a4-9cac-10497bf8ee09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:34.243898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b6b9e-03fa-49a4-9cac-10497bf8ee09 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:34.244068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b6b9e-03fa-49a4-9cac-10497bf8ee09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:34.244344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b6b9e-03fa-49a4-9cac-10497bf8ee09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:34.245032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b6b9e-03fa-49a4-9cac-10497bf8ee09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:34.248823 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b6b9e-03fa-49a4-9cac-10497bf8ee09 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:34.263497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 945/1863] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:34.264295 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:34.264537 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b6b9e-03fa-49a4-9cac-10497bf8ee09 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:34.270042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63598403-9870-46a9-a008-878975c7a950 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:34.273668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f7462-1ade-4773-b0ca-24815d8c7d91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:34.275448 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f7462-1ade-4773-b0ca-24815d8c7d91 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:34.275448 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f7462-1ade-4773-b0ca-24815d8c7d91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:34.275448 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f7462-1ade-4773-b0ca-24815d8c7d91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:34.275657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b6b9e-03fa-49a4-9cac-10497bf8ee09 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:34.279563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3dc06c-5f5a-424b-acc4-ff3e65a4b503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:34.280149 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dc06c-5f5a-424b-acc4-ff3e65a4b503 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:34.280430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dc06c-5f5a-424b-acc4-ff3e65a4b503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:34.280739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dc06c-5f5a-424b-acc4-ff3e65a4b503 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:34.281375 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3dc06c-5f5a-424b-acc4-ff3e65a4b503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:34.283177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3dc06c-5f5a-424b-acc4-ff3e65a4b503 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:34.284776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f7462-1ade-4773-b0ca-24815d8c7d91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:34.286189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f7462-1ade-4773-b0ca-24815d8c7d91 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:34.289249 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:34.295441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 946/1864] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:34.296617 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 919/1865] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:19:34.299865 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3dc06c-5f5a-424b-acc4-ff3e65a4b503 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:34.309762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3dc06c-5f5a-424b-acc4-ff3e65a4b503 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:34.315135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c1dbb-22be-4a49-bd66-539383bb8575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:34.316144 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c1dbb-22be-4a49-bd66-539383bb8575 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:34.316837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c1dbb-22be-4a49-bd66-539383bb8575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:34.317232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c1dbb-22be-4a49-bd66-539383bb8575 admin admin] PATH_INFO: `/v3/projects/2665ff2a74e342ed812f95732dba6016/users/012bd308c2c24589b9885a035d57dc2a/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:34.332318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c1dbb-22be-4a49-bd66-539383bb8575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2665ff2a74e342ed812f95732dba6016, user_id=012bd308c2c24589b9885a035d57dc2a, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:34.335335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c1dbb-22be-4a49-bd66-539383bb8575 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:34.438609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 947/1866] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:34 2022] PUT /identity/v3/projects/2665ff2a74e342ed812f95732dba6016/users/012bd308c2c24589b9885a035d57dc2a/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:34.450874 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22264974-dac3-4942-80e5-47587108509b None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:34.451169 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22264974-dac3-4942-80e5-47587108509b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:34.451468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22264974-dac3-4942-80e5-47587108509b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:34.479706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-22264974-dac3-4942-80e5-47587108509b None None] MFA Rules not processed for user `012bd308c2c24589b9885a035d57dc2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:34.480953 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-22264974-dac3-4942-80e5-47587108509b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:34.559718 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 920/1867] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:34.564987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c1dbb-22be-4a49-bd66-539383bb8575 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:34.636765 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c1dbb-22be-4a49-bd66-539383bb8575 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:34.643473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b2e9a-007b-40cd-842a-d201b2023563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:34.644343 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b2e9a-007b-40cd-842a-d201b2023563 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:34.644631 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b2e9a-007b-40cd-842a-d201b2023563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:34.644944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b2e9a-007b-40cd-842a-d201b2023563 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:34.645762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b2e9a-007b-40cd-842a-d201b2023563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:34.647587 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b2e9a-007b-40cd-842a-d201b2023563 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:34.726366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22264974-dac3-4942-80e5-47587108509b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:34.727853 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-22264974-dac3-4942-80e5-47587108509b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:34.741890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22264974-dac3-4942-80e5-47587108509b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:34.746617 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95560cee-b499-41c3-88bc-ea40ebbdc125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:34.747248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95560cee-b499-41c3-88bc-ea40ebbdc125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:34.747530 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95560cee-b499-41c3-88bc-ea40ebbdc125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:34.747799 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95560cee-b499-41c3-88bc-ea40ebbdc125 admin admin] PATH_INFO: `/v3/projects/9c4e7545107a456fb285f0918b0fd30f` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:34.748953 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95560cee-b499-41c3-88bc-ea40ebbdc125 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4e7545107a456fb285f0918b0fd30f)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:34.750889 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95560cee-b499-41c3-88bc-ea40ebbdc125 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:34.766383 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 948/1868] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:34 2022] POST /identity/v3/projects => generated 375 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:34.770645 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b2e9a-007b-40cd-842a-d201b2023563 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:34.834535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b2e9a-007b-40cd-842a-d201b2023563 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:34.840171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d6f85-c907-4fe5-a134-6b6121b540c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:34.840895 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d6f85-c907-4fe5-a134-6b6121b540c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:34.841250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d6f85-c907-4fe5-a134-6b6121b540c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:34.841531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d6f85-c907-4fe5-a134-6b6121b540c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:34.842187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d6f85-c907-4fe5-a134-6b6121b540c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:34.844597 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d6f85-c907-4fe5-a134-6b6121b540c2 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:34.859161 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-95560cee-b499-41c3-88bc-ea40ebbdc125 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4e7545107a456fb285f0918b0fd30f', 'request_id': 'req-95560cee-b499-41c3-88bc-ea40ebbdc125'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:34.866069 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-95560cee-b499-41c3-88bc-ea40ebbdc125 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4e7545107a456fb285f0918b0fd30f', 'request_id': 'req-95560cee-b499-41c3-88bc-ea40ebbdc125'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:34.872731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-95560cee-b499-41c3-88bc-ea40ebbdc125 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4e7545107a456fb285f0918b0fd30f', 'request_id': 'req-95560cee-b499-41c3-88bc-ea40ebbdc125'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:35.010118 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-95560cee-b499-41c3-88bc-ea40ebbdc125 admin admin] The token cache is being invalidate because project 9c4e7545107a456fb285f0918b0fd30f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:35.010418 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-95560cee-b499-41c3-88bc-ea40ebbdc125 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:35.011135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-95560cee-b499-41c3-88bc-ea40ebbdc125 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:35.012889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 949/1869] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:34 2022] POST /identity/v3/users => generated 288 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:19:35.013022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 921/1870] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:34 2022] DELETE /identity/v3/projects/9c4e7545107a456fb285f0918b0fd30f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:19:35.022367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d6f85-c907-4fe5-a134-6b6121b540c2 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:35.022903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5066f19-5a14-4323-8bcc-e1dcca72421d None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:35.023197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5066f19-5a14-4323-8bcc-e1dcca72421d None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:35.023541 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5066f19-5a14-4323-8bcc-e1dcca72421d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:35.024718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-415d6f85-c907-4fe5-a134-6b6121b540c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:35.037987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d6f85-c907-4fe5-a134-6b6121b540c2 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:35.045458 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-f5066f19-5a14-4323-8bcc-e1dcca72421d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:35.046796 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f5066f19-5a14-4323-8bcc-e1dcca72421d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:35.107130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:35.124156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27398a-2376-4c7a-a9fe-99da04cd7fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:35.124947 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27398a-2376-4c7a-a9fe-99da04cd7fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:35.125705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27398a-2376-4c7a-a9fe-99da04cd7fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:35.126014 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27398a-2376-4c7a-a9fe-99da04cd7fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:35.127531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e27398a-2376-4c7a-a9fe-99da04cd7fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:35.128811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 922/1871] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:35.130479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e27398a-2376-4c7a-a9fe-99da04cd7fb1 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:35.133656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc8e10-21c7-4434-a04b-6583adbc685f None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:35.134077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc8e10-21c7-4434-a04b-6583adbc685f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:35.134525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc8e10-21c7-4434-a04b-6583adbc685f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:35.143516 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:35.150431 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 950/1872] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:19:35.151390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27398a-2376-4c7a-a9fe-99da04cd7fb1 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:35.162899 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-e3dc8e10-21c7-4434-a04b-6583adbc685f None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:35.164932 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-e3dc8e10-21c7-4434-a04b-6583adbc685f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:35.165921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27398a-2376-4c7a-a9fe-99da04cd7fb1 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:35.172706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2000fe-e6b0-49d4-9b0c-619734634492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:35.174083 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2000fe-e6b0-49d4-9b0c-619734634492 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:35.174640 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2000fe-e6b0-49d4-9b0c-619734634492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:35.178682 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2000fe-e6b0-49d4-9b0c-619734634492 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:35.178682 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2000fe-e6b0-49d4-9b0c-619734634492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:35.179465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 923/1873] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:35.180230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2000fe-e6b0-49d4-9b0c-619734634492 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:35.186182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc8e10-21c7-4434-a04b-6583adbc685f None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:35.192894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 951/1874] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:35.196438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc8e10-21c7-4434-a04b-6583adbc685f None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:35.197095 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2000fe-e6b0-49d4-9b0c-619734634492 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:35.211285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2000fe-e6b0-49d4-9b0c-619734634492 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:35.216426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14836381-51cc-4a02-895a-d04a5b5f2c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:35.217281 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14836381-51cc-4a02-895a-d04a5b5f2c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:35.217757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14836381-51cc-4a02-895a-d04a5b5f2c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:35.218159 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14836381-51cc-4a02-895a-d04a5b5f2c42 admin admin] PATH_INFO: `/v3/projects/37124e29df3e42aa847fa25d41f9d5fa/users/727e03f63a21442380fce6f4b3824c83/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:35.228706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77bc74-8f55-4d97-bbd6-ca6649275d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:35.229401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77bc74-8f55-4d97-bbd6-ca6649275d0f admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:35.229780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77bc74-8f55-4d97-bbd6-ca6649275d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:35.230259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77bc74-8f55-4d97-bbd6-ca6649275d0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:35.230932 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77bc74-8f55-4d97-bbd6-ca6649275d0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:19:35.232143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77bc74-8f55-4d97-bbd6-ca6649275d0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:35.234202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77bc74-8f55-4d97-bbd6-ca6649275d0f admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:35.240900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14836381-51cc-4a02-895a-d04a5b5f2c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37124e29df3e42aa847fa25d41f9d5fa, user_id=727e03f63a21442380fce6f4b3824c83, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:35.244305 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14836381-51cc-4a02-895a-d04a5b5f2c42 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:35.249173 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 924/1875] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:19:35.250209 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77bc74-8f55-4d97-bbd6-ca6649275d0f admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:35.252722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2d77bc74-8f55-4d97-bbd6-ca6649275d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:35.313125 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77bc74-8f55-4d97-bbd6-ca6649275d0f admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:35.322979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 952/1876] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:35 2022] PUT /identity/v3/projects/37124e29df3e42aa847fa25d41f9d5fa/users/727e03f63a21442380fce6f4b3824c83/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:19:35.324082 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14836381-51cc-4a02-895a-d04a5b5f2c42 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:35.410116 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654125d0-1abc-4163-ab49-91e24eb5fcff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:35.411224 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654125d0-1abc-4163-ab49-91e24eb5fcff service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:35.411568 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654125d0-1abc-4163-ab49-91e24eb5fcff service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:35.411910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654125d0-1abc-4163-ab49-91e24eb5fcff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:35.413019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14836381-51cc-4a02-895a-d04a5b5f2c42 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:35.414810 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-654125d0-1abc-4163-ab49-91e24eb5fcff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:35.418629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1db725-6f39-44b9-9f4c-3d16d569b7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:35.419179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1db725-6f39-44b9-9f4c-3d16d569b7ae admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:35.419493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1db725-6f39-44b9-9f4c-3d16d569b7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:35.420677 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1db725-6f39-44b9-9f4c-3d16d569b7ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:35.420677 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1db725-6f39-44b9-9f4c-3d16d569b7ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:35.427593 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1db725-6f39-44b9-9f4c-3d16d569b7ae admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:35.507679 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654125d0-1abc-4163-ab49-91e24eb5fcff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:35.509149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654125d0-1abc-4163-ab49-91e24eb5fcff service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:35.520737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 925/1877] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:35.521871 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af7ca9-9e1a-416d-858d-40d75eb1cf45 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:35.522156 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af7ca9-9e1a-416d-858d-40d75eb1cf45 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:35.522411 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af7ca9-9e1a-416d-858d-40d75eb1cf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:35.571076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-e2af7ca9-9e1a-416d-858d-40d75eb1cf45 None None] MFA Rules not processed for user `727e03f63a21442380fce6f4b3824c83`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:35.572505 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-e2af7ca9-9e1a-416d-858d-40d75eb1cf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:35.574459 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 953/1878] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:35 2022] POST /identity/v3/projects => generated 385 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:35.578450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1db725-6f39-44b9-9f4c-3d16d569b7ae admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:35.620146 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:35.645663 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1db725-6f39-44b9-9f4c-3d16d569b7ae admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:35.646891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 926/1879] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:35.650626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54db9b1-41d8-4f30-8912-2f0c9148e675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:35.651318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54db9b1-41d8-4f30-8912-2f0c9148e675 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:35.651428 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af7ca9-9e1a-416d-858d-40d75eb1cf45 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:35.651579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54db9b1-41d8-4f30-8912-2f0c9148e675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:35.651851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54db9b1-41d8-4f30-8912-2f0c9148e675 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:35.652462 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54db9b1-41d8-4f30-8912-2f0c9148e675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:35.654413 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54db9b1-41d8-4f30-8912-2f0c9148e675 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:35.662231 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af7ca9-9e1a-416d-858d-40d75eb1cf45 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:35.668575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19280bc9-906c-452c-ba85-c98b380a1729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:35.668575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19280bc9-906c-452c-ba85-c98b380a1729 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:35.668575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19280bc9-906c-452c-ba85-c98b380a1729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:35.668894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19280bc9-906c-452c-ba85-c98b380a1729 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:35.669488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19280bc9-906c-452c-ba85-c98b380a1729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:35.671402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19280bc9-906c-452c-ba85-c98b380a1729 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:35.753476 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 954/1880] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:35 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:35.757793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54db9b1-41d8-4f30-8912-2f0c9148e675 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:35.769052 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54db9b1-41d8-4f30-8912-2f0c9148e675 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:35.774562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a21e09-8212-40b7-931f-da204f66bedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:35.775282 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a21e09-8212-40b7-931f-da204f66bedb admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:35.776410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a21e09-8212-40b7-931f-da204f66bedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:35.776410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a21e09-8212-40b7-931f-da204f66bedb admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:35.777279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a21e09-8212-40b7-931f-da204f66bedb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:35.779537 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a21e09-8212-40b7-931f-da204f66bedb admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:35.793782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 955/1881] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:19:35.793958 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 927/1882] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:35 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:19:35.796817 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19280bc9-906c-452c-ba85-c98b380a1729 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:35.801687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a21e09-8212-40b7-931f-da204f66bedb admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:35.814240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a21e09-8212-40b7-931f-da204f66bedb admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:35.827554 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:35.899369 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19280bc9-906c-452c-ba85-c98b380a1729 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:35.909563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e67948-9c7b-4a01-b2b3-8f06e9734c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:35.910172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e67948-9c7b-4a01-b2b3-8f06e9734c00 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:35.910477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e67948-9c7b-4a01-b2b3-8f06e9734c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:35.910749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e67948-9c7b-4a01-b2b3-8f06e9734c00 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:35.911371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e67948-9c7b-4a01-b2b3-8f06e9734c00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:35.913480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e67948-9c7b-4a01-b2b3-8f06e9734c00 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:35.913684 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fed6fc-acc9-422f-8be4-6019c66af818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:35.914318 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fed6fc-acc9-422f-8be4-6019c66af818 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:35.914860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fed6fc-acc9-422f-8be4-6019c66af818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:35.915134 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fed6fc-acc9-422f-8be4-6019c66af818 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:35.915794 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fed6fc-acc9-422f-8be4-6019c66af818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:35.917692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fed6fc-acc9-422f-8be4-6019c66af818 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:35.929379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 928/1883] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:35.933510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fed6fc-acc9-422f-8be4-6019c66af818 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:35.944031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fed6fc-acc9-422f-8be4-6019c66af818 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:35.949046 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564a225-0f89-4c78-948b-7bf25dafbc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:35.955167 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564a225-0f89-4c78-948b-7bf25dafbc41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:35.955167 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564a225-0f89-4c78-948b-7bf25dafbc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:35.955167 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564a225-0f89-4c78-948b-7bf25dafbc41 admin admin] PATH_INFO: `/v3/projects/88f4251319c843aca548bf3546a12b2e/users/f9c2a4e4989b42348010571c6103dc50/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:35.967300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564a225-0f89-4c78-948b-7bf25dafbc41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f4251319c843aca548bf3546a12b2e, user_id=f9c2a4e4989b42348010571c6103dc50, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:35.970645 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564a225-0f89-4c78-948b-7bf25dafbc41 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:36.012506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 956/1884] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:35 2022] POST /identity/v3/users => generated 295 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:36.014566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e67948-9c7b-4a01-b2b3-8f06e9734c00 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:36.024713 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e67948-9c7b-4a01-b2b3-8f06e9734c00 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:36.029226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a7949-44a2-4efd-8eae-e63fb82ff1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:36.029794 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a7949-44a2-4efd-8eae-e63fb82ff1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:36.030073 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a7949-44a2-4efd-8eae-e63fb82ff1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:36.030343 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a7949-44a2-4efd-8eae-e63fb82ff1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:36.031012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a7949-44a2-4efd-8eae-e63fb82ff1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:36.032668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a7949-44a2-4efd-8eae-e63fb82ff1a7 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:36.044193 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 957/1885] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:19:36.048280 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a7949-44a2-4efd-8eae-e63fb82ff1a7 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:36.057320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a7949-44a2-4efd-8eae-e63fb82ff1a7 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:36.061925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dfef9-9704-4f98-b12f-6f5daa03f3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:36.062540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dfef9-9704-4f98-b12f-6f5daa03f3c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:36.062871 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dfef9-9704-4f98-b12f-6f5daa03f3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:36.063150 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dfef9-9704-4f98-b12f-6f5daa03f3c4 admin admin] PATH_INFO: `/v3/projects/91e0223835e344aea8c5c266ae012a08/users/73734ac1ae9a42d797f2f0c913534ba0/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:36.067943 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 929/1886] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:35 2022] PUT /identity/v3/projects/88f4251319c843aca548bf3546a12b2e/users/f9c2a4e4989b42348010571c6103dc50/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:19:36.078852 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928dfef9-9704-4f98-b12f-6f5daa03f3c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e0223835e344aea8c5c266ae012a08, user_id=73734ac1ae9a42d797f2f0c913534ba0, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:36.082546 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928dfef9-9704-4f98-b12f-6f5daa03f3c4 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:36.086016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0143c8-84e6-4487-8b36-b92c872c2585 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:36.086302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0143c8-84e6-4487-8b36-b92c872c2585 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:36.086872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0143c8-84e6-4487-8b36-b92c872c2585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:36.114963 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-cb0143c8-84e6-4487-8b36-b92c872c2585 None None] MFA Rules not processed for user `f9c2a4e4989b42348010571c6103dc50`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:36.116166 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-cb0143c8-84e6-4487-8b36-b92c872c2585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:36.159109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 958/1887] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:36 2022] PUT /identity/v3/projects/91e0223835e344aea8c5c266ae012a08/users/73734ac1ae9a42d797f2f0c913534ba0/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:19:36.163294 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dfef9-9704-4f98-b12f-6f5daa03f3c4 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:36.184047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 930/1888] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:36.196228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0143c8-84e6-4487-8b36-b92c872c2585 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:36.198265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-cb0143c8-84e6-4487-8b36-b92c872c2585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:36.222142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:36.239183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dfef9-9704-4f98-b12f-6f5daa03f3c4 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:36.244243 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf6441-7d69-4fad-a70e-7fa7992c6ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:36.244882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf6441-7d69-4fad-a70e-7fa7992c6ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:36.247381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf6441-7d69-4fad-a70e-7fa7992c6ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:36.247381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf6441-7d69-4fad-a70e-7fa7992c6ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:36.247381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf6441-7d69-4fad-a70e-7fa7992c6ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:36.247769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf6441-7d69-4fad-a70e-7fa7992c6ed9 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:36.258519 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 959/1889] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:19:36.261933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0143c8-84e6-4487-8b36-b92c872c2585 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:36.262798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf6441-7d69-4fad-a70e-7fa7992c6ed9 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:36.272463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf6441-7d69-4fad-a70e-7fa7992c6ed9 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:36.277397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffd329-2ec6-473f-a48d-fdffa6bab148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:36.278140 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffd329-2ec6-473f-a48d-fdffa6bab148 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:36.278488 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffd329-2ec6-473f-a48d-fdffa6bab148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:36.278777 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffd329-2ec6-473f-a48d-fdffa6bab148 admin admin] PATH_INFO: `/v3/projects/91e0223835e344aea8c5c266ae012a08/users/73734ac1ae9a42d797f2f0c913534ba0/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:36.280779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ffd329-2ec6-473f-a48d-fdffa6bab148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e0223835e344aea8c5c266ae012a08, user_id=73734ac1ae9a42d797f2f0c913534ba0, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:36.283690 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ffd329-2ec6-473f-a48d-fdffa6bab148 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:36.294395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 960/1890] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:36 2022] PUT /identity/v3/projects/91e0223835e344aea8c5c266ae012a08/users/73734ac1ae9a42d797f2f0c913534ba0/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:36.306053 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13204ebf-1c68-40f1-8672-b7099c5391e1 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:36.306339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13204ebf-1c68-40f1-8672-b7099c5391e1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:36.310984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13204ebf-1c68-40f1-8672-b7099c5391e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:36.311101 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179500b5-e0d3-4aeb-9b3d-1549ea4df2ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:36.311101 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179500b5-e0d3-4aeb-9b3d-1549ea4df2ef service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:36.311350 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179500b5-e0d3-4aeb-9b3d-1549ea4df2ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:36.311623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179500b5-e0d3-4aeb-9b3d-1549ea4df2ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:36.336247 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-13204ebf-1c68-40f1-8672-b7099c5391e1 None None] MFA Rules not processed for user `73734ac1ae9a42d797f2f0c913534ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:36.337486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-13204ebf-1c68-40f1-8672-b7099c5391e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:36.394850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179500b5-e0d3-4aeb-9b3d-1549ea4df2ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:36.397235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179500b5-e0d3-4aeb-9b3d-1549ea4df2ef service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:36.400535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:36.407232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 931/1891] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:19:36.442810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 961/1892] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:36.464497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179500b5-e0d3-4aeb-9b3d-1549ea4df2ef service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:36.479275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179500b5-e0d3-4aeb-9b3d-1549ea4df2ef service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:36.540063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1587279-bd86-44fb-959f-abe2a260e19c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:36.540575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1587279-bd86-44fb-959f-abe2a260e19c service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:36.540888 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1587279-bd86-44fb-959f-abe2a260e19c service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:36.541164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1587279-bd86-44fb-959f-abe2a260e19c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:36.551075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1587279-bd86-44fb-959f-abe2a260e19c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:36.552500 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1587279-bd86-44fb-959f-abe2a260e19c service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:36.565789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 932/1893] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:36.693875 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13204ebf-1c68-40f1-8672-b7099c5391e1 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:36.704066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13204ebf-1c68-40f1-8672-b7099c5391e1 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:36.708755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98ccf3-bb2d-484a-bd9b-8e04eb9cbf4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:36.709400 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98ccf3-bb2d-484a-bd9b-8e04eb9cbf4a service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:36.709705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98ccf3-bb2d-484a-bd9b-8e04eb9cbf4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:36.710009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98ccf3-bb2d-484a-bd9b-8e04eb9cbf4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:36.711842 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-8a98ccf3-bb2d-484a-bd9b-8e04eb9cbf4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:36.777172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98ccf3-bb2d-484a-bd9b-8e04eb9cbf4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:36.778652 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98ccf3-bb2d-484a-bd9b-8e04eb9cbf4a service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:36.791546 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 962/1894] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:38.168291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1587279-bd86-44fb-959f-abe2a260e19c service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:38.176905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1587279-bd86-44fb-959f-abe2a260e19c service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:38.180710 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cb822-b25e-4524-a5cb-d0d0df3a2379 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:38.181378 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cb822-b25e-4524-a5cb-d0d0df3a2379 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:38.181745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cb822-b25e-4524-a5cb-d0d0df3a2379 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:38.182075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cb822-b25e-4524-a5cb-d0d0df3a2379 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:38.183921 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a24cb822-b25e-4524-a5cb-d0d0df3a2379 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:38.217561 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:38.246462 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cb822-b25e-4524-a5cb-d0d0df3a2379 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:38.247923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cb822-b25e-4524-a5cb-d0d0df3a2379 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:38.257677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 933/1895] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:44.638831 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ee733-609e-47c4-b14b-c6894b7db622 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:44.639361 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ee733-609e-47c4-b14b-c6894b7db622 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:44.639361 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ee733-609e-47c4-b14b-c6894b7db622 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:44.640574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 963/1896] 10.222.0.27 () {54 vars in 921 bytes} [Fri Aug 12 12:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:19:44.652338 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889e13c-bcbb-4c47-b2c3-e1998f5ce82b None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:44.652885 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889e13c-bcbb-4c47-b2c3-e1998f5ce82b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:44.652885 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889e13c-bcbb-4c47-b2c3-e1998f5ce82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:44.674223 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-6889e13c-bcbb-4c47-b2c3-e1998f5ce82b None None] MFA Rules not processed for user `687d0375abd44d2481f988c6df90fad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:44.674575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6889e13c-bcbb-4c47-b2c3-e1998f5ce82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:44.742607 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 934/1897] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Aug 12 12:19:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:44.754893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ee733-609e-47c4-b14b-c6894b7db622 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:44.756593 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b55ee733-609e-47c4-b14b-c6894b7db622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:44.791083 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ee733-609e-47c4-b14b-c6894b7db622 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:44.801096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7f098-ee40-4fd2-9e2c-ec1fc6fc8560 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:44.801762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7f098-ee40-4fd2-9e2c-ec1fc6fc8560 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:44.802043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7f098-ee40-4fd2-9e2c-ec1fc6fc8560 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:44.802316 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7f098-ee40-4fd2-9e2c-ec1fc6fc8560 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:44.819835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7f098-ee40-4fd2-9e2c-ec1fc6fc8560 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:44.821601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7f098-ee40-4fd2-9e2c-ec1fc6fc8560 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:44.832259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 964/1898] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:45.245736 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889e13c-bcbb-4c47-b2c3-e1998f5ce82b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:45.328551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889e13c-bcbb-4c47-b2c3-e1998f5ce82b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:45.334177 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a31bd-3c92-4995-8794-5f7c35bb8ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:45.335088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a31bd-3c92-4995-8794-5f7c35bb8ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:45.335366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a31bd-3c92-4995-8794-5f7c35bb8ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:45.335787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a31bd-3c92-4995-8794-5f7c35bb8ba9 admin admin] PATH_INFO: `/v3/users/012bd308c2c24589b9885a035d57dc2a` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:45.337284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a31bd-3c92-4995-8794-5f7c35bb8ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=012bd308c2c24589b9885a035d57dc2a)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:45.339391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a31bd-3c92-4995-8794-5f7c35bb8ba9 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:45.600316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4f7a31bd-3c92-4995-8794-5f7c35bb8ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '012bd308c2c24589b9885a035d57dc2a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:45.601044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4f7a31bd-3c92-4995-8794-5f7c35bb8ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '012bd308c2c24589b9885a035d57dc2a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:45.754161 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4f7a31bd-3c92-4995-8794-5f7c35bb8ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '012bd308c2c24589b9885a035d57dc2a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:45.768971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:45.770477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4f7a31bd-3c92-4995-8794-5f7c35bb8ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '012bd308c2c24589b9885a035d57dc2a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:45.777650 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4f7a31bd-3c92-4995-8794-5f7c35bb8ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '012bd308c2c24589b9885a035d57dc2a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:45.783725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 935/1899] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:45 2022] DELETE /identity/v3/users/012bd308c2c24589b9885a035d57dc2a => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:45.788322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042b412-9dc9-4a3f-892e-32060115bc31 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:45.788837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042b412-9dc9-4a3f-892e-32060115bc31 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:45.788925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042b412-9dc9-4a3f-892e-32060115bc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:45.808089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-2042b412-9dc9-4a3f-892e-32060115bc31 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:45.809311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2042b412-9dc9-4a3f-892e-32060115bc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:45.868726 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 965/1900] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:45.879399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a31bd-3c92-4995-8794-5f7c35bb8ba9 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:45.880854 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4f7a31bd-3c92-4995-8794-5f7c35bb8ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:45.937579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a31bd-3c92-4995-8794-5f7c35bb8ba9 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:45.941928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88bd70-b61d-41bd-9369-7300e09d4390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:45.942524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88bd70-b61d-41bd-9369-7300e09d4390 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:45.942804 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88bd70-b61d-41bd-9369-7300e09d4390 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:45.943073 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88bd70-b61d-41bd-9369-7300e09d4390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:45.951840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88bd70-b61d-41bd-9369-7300e09d4390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:45.953288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88bd70-b61d-41bd-9369-7300e09d4390 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:45.962629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 936/1901] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:46.076338 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042b412-9dc9-4a3f-892e-32060115bc31 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:46.077827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2042b412-9dc9-4a3f-892e-32060115bc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:46.089832 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042b412-9dc9-4a3f-892e-32060115bc31 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:46.094466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d785297-b28d-4100-939c-1f9e323554f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:46.095092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d785297-b28d-4100-939c-1f9e323554f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:46.095375 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d785297-b28d-4100-939c-1f9e323554f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:46.095652 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d785297-b28d-4100-939c-1f9e323554f1 admin admin] PATH_INFO: `/v3/users/19511af5527940daaf50aaf01dd5ce43` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:46.096794 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d785297-b28d-4100-939c-1f9e323554f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19511af5527940daaf50aaf01dd5ce43)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:46.098708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d785297-b28d-4100-939c-1f9e323554f1 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:46.283268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5d785297-b28d-4100-939c-1f9e323554f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19511af5527940daaf50aaf01dd5ce43'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:46.354018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5d785297-b28d-4100-939c-1f9e323554f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19511af5527940daaf50aaf01dd5ce43'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:46.354685 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5d785297-b28d-4100-939c-1f9e323554f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19511af5527940daaf50aaf01dd5ce43'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:46.362592 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:46.363826 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5d785297-b28d-4100-939c-1f9e323554f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19511af5527940daaf50aaf01dd5ce43'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:46.375421 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5d785297-b28d-4100-939c-1f9e323554f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19511af5527940daaf50aaf01dd5ce43'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:46.382056 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 966/1902] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:46 2022] DELETE /identity/v3/users/19511af5527940daaf50aaf01dd5ce43 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:46.388985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88bd70-b61d-41bd-9369-7300e09d4390 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:46.390643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-5e88bd70-b61d-41bd-9369-7300e09d4390 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:46.391183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f3e6a-1058-4dc7-ab0c-f4e23ee61ba4 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:46.392286 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f3e6a-1058-4dc7-ab0c-f4e23ee61ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:46.392488 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f3e6a-1058-4dc7-ab0c-f4e23ee61ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:46.411466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88bd70-b61d-41bd-9369-7300e09d4390 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:46.415926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-aa3f3e6a-1058-4dc7-ab0c-f4e23ee61ba4 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:46.417514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-aa3f3e6a-1058-4dc7-ab0c-f4e23ee61ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:46.470197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:46.470786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:46.471074 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:46.471351 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1 admin admin] PATH_INFO: `/v3/projects/2665ff2a74e342ed812f95732dba6016` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:46.472478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2665ff2a74e342ed812f95732dba6016)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:46.474551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:46.478111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 967/1903] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:46.487966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f3e6a-1058-4dc7-ab0c-f4e23ee61ba4 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:46.489349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-aa3f3e6a-1058-4dc7-ab0c-f4e23ee61ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:46.546550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f3e6a-1058-4dc7-ab0c-f4e23ee61ba4 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:46.550809 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444583f3-7588-4672-8d27-683d6e8f5dad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:46.551405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444583f3-7588-4672-8d27-683d6e8f5dad service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:46.551684 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444583f3-7588-4672-8d27-683d6e8f5dad service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:46.551962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444583f3-7588-4672-8d27-683d6e8f5dad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:46.560972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444583f3-7588-4672-8d27-683d6e8f5dad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:46.562388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444583f3-7588-4672-8d27-683d6e8f5dad service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:46.580908 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 968/1904] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:19:46.609933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2665ff2a74e342ed812f95732dba6016', 'request_id': 'req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:46.617137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2665ff2a74e342ed812f95732dba6016', 'request_id': 'req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:46.624392 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2665ff2a74e342ed812f95732dba6016', 'request_id': 'req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:46.726123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1 admin admin] The token cache is being invalidate because project 2665ff2a74e342ed812f95732dba6016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:46.726416 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:46.729476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:46.729476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 937/1905] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:46 2022] DELETE /identity/v3/projects/2665ff2a74e342ed812f95732dba6016 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:46.732572 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444583f3-7588-4672-8d27-683d6e8f5dad service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:46.734029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-444583f3-7588-4672-8d27-683d6e8f5dad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:46.747963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444583f3-7588-4672-8d27-683d6e8f5dad service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:46.852119 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee8215-5210-414c-baee-f0d2cfac5f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:46.852824 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee8215-5210-414c-baee-f0d2cfac5f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:46.853108 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee8215-5210-414c-baee-f0d2cfac5f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:46.853387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee8215-5210-414c-baee-f0d2cfac5f2c admin admin] PATH_INFO: `/v3/users/727e03f63a21442380fce6f4b3824c83` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:46.854568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee8215-5210-414c-baee-f0d2cfac5f2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=727e03f63a21442380fce6f4b3824c83)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:46.856559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee8215-5210-414c-baee-f0d2cfac5f2c admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:46.882184 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:47.102369 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:47.103998 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:47.119257 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c486d-6c7a-45bd-bc86-e12898d15bf1 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:47.125149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554b800-bec3-44e6-b870-591b4dcee464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:47.126276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554b800-bec3-44e6-b870-591b4dcee464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:47.127158 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554b800-bec3-44e6-b870-591b4dcee464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:47.127592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554b800-bec3-44e6-b870-591b4dcee464 admin admin] PATH_INFO: `/v3/projects/8865934723c34e17b1237525a10f28ff` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:47.128688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f5ee8215-5210-414c-baee-f0d2cfac5f2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '727e03f63a21442380fce6f4b3824c83'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.129288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554b800-bec3-44e6-b870-591b4dcee464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8865934723c34e17b1237525a10f28ff)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:47.131402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554b800-bec3-44e6-b870-591b4dcee464 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:47.140309 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:47.217371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f5ee8215-5210-414c-baee-f0d2cfac5f2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '727e03f63a21442380fce6f4b3824c83'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.217995 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f5ee8215-5210-414c-baee-f0d2cfac5f2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '727e03f63a21442380fce6f4b3824c83'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.226397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f5ee8215-5210-414c-baee-f0d2cfac5f2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '727e03f63a21442380fce6f4b3824c83'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.239470 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f5ee8215-5210-414c-baee-f0d2cfac5f2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '727e03f63a21442380fce6f4b3824c83'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.244661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a554b800-bec3-44e6-b870-591b4dcee464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8865934723c34e17b1237525a10f28ff', 'request_id': 'req-a554b800-bec3-44e6-b870-591b4dcee464'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.249667 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 969/1906] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:46 2022] DELETE /identity/v3/users/727e03f63a21442380fce6f4b3824c83 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:47.250551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a554b800-bec3-44e6-b870-591b4dcee464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8865934723c34e17b1237525a10f28ff', 'request_id': 'req-a554b800-bec3-44e6-b870-591b4dcee464'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.258971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a554b800-bec3-44e6-b870-591b4dcee464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8865934723c34e17b1237525a10f28ff', 'request_id': 'req-a554b800-bec3-44e6-b870-591b4dcee464'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.322458 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a554b800-bec3-44e6-b870-591b4dcee464 admin admin] The token cache is being invalidate because project 8865934723c34e17b1237525a10f28ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:47.322758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a554b800-bec3-44e6-b870-591b4dcee464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.323414 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a554b800-bec3-44e6-b870-591b4dcee464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.324994 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 938/1907] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:47 2022] DELETE /identity/v3/projects/8865934723c34e17b1237525a10f28ff => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:19:47.329990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee8215-5210-414c-baee-f0d2cfac5f2c admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:47.331455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f5ee8215-5210-414c-baee-f0d2cfac5f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:47.398997 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee8215-5210-414c-baee-f0d2cfac5f2c admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:47.403646 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8130f0-000e-48bb-8777-60b242090361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:47.404358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8130f0-000e-48bb-8777-60b242090361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:47.404666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8130f0-000e-48bb-8777-60b242090361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:47.404961 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8130f0-000e-48bb-8777-60b242090361 admin admin] PATH_INFO: `/v3/users/c086e8fa2f8141b29a3d8b35272c7f3c` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:47.406097 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8130f0-000e-48bb-8777-60b242090361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c086e8fa2f8141b29a3d8b35272c7f3c)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:47.409002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8130f0-000e-48bb-8777-60b242090361 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:47.628214 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4e8130f0-000e-48bb-8777-60b242090361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c086e8fa2f8141b29a3d8b35272c7f3c'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.662992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554b800-bec3-44e6-b870-591b4dcee464 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:47.664837 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a554b800-bec3-44e6-b870-591b4dcee464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:47.682871 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4e8130f0-000e-48bb-8777-60b242090361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c086e8fa2f8141b29a3d8b35272c7f3c'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.683684 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4e8130f0-000e-48bb-8777-60b242090361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c086e8fa2f8141b29a3d8b35272c7f3c'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.694838 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4e8130f0-000e-48bb-8777-60b242090361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c086e8fa2f8141b29a3d8b35272c7f3c'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.710398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4e8130f0-000e-48bb-8777-60b242090361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c086e8fa2f8141b29a3d8b35272c7f3c'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.717104 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 970/1908] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:47 2022] DELETE /identity/v3/users/c086e8fa2f8141b29a3d8b35272c7f3c => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:47.741142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554b800-bec3-44e6-b870-591b4dcee464 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:47.803232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7aa50-468f-44be-ab3c-6c730ece36ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:47.803972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7aa50-468f-44be-ab3c-6c730ece36ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:47.804792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7aa50-468f-44be-ab3c-6c730ece36ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:47.805140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7aa50-468f-44be-ab3c-6c730ece36ff admin admin] PATH_INFO: `/v3/projects/37124e29df3e42aa847fa25d41f9d5fa` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:47.806372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c7aa50-468f-44be-ab3c-6c730ece36ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=37124e29df3e42aa847fa25d41f9d5fa)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:47.810183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c7aa50-468f-44be-ab3c-6c730ece36ff admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:47.927626 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:47.930582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e0c7aa50-468f-44be-ab3c-6c730ece36ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37124e29df3e42aa847fa25d41f9d5fa', 'request_id': 'req-e0c7aa50-468f-44be-ab3c-6c730ece36ff'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.936351 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e0c7aa50-468f-44be-ab3c-6c730ece36ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37124e29df3e42aa847fa25d41f9d5fa', 'request_id': 'req-e0c7aa50-468f-44be-ab3c-6c730ece36ff'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:47.943270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e0c7aa50-468f-44be-ab3c-6c730ece36ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37124e29df3e42aa847fa25d41f9d5fa', 'request_id': 'req-e0c7aa50-468f-44be-ab3c-6c730ece36ff'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.017105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e0c7aa50-468f-44be-ab3c-6c730ece36ff admin admin] The token cache is being invalidate because project 37124e29df3e42aa847fa25d41f9d5fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:48.017475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e0c7aa50-468f-44be-ab3c-6c730ece36ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.018945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e0c7aa50-468f-44be-ab3c-6c730ece36ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.020670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 939/1909] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:47 2022] DELETE /identity/v3/projects/37124e29df3e42aa847fa25d41f9d5fa => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:19:48.024364 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8130f0-000e-48bb-8777-60b242090361 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:48.026081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4e8130f0-000e-48bb-8777-60b242090361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:48.041550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8130f0-000e-48bb-8777-60b242090361 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:48.057191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:48.141061 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cceb0-6e01-4543-b6df-18a6b2deba11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:48.141818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cceb0-6e01-4543-b6df-18a6b2deba11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:48.142146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cceb0-6e01-4543-b6df-18a6b2deba11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:48.142436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cceb0-6e01-4543-b6df-18a6b2deba11 admin admin] PATH_INFO: `/v3/users/73734ac1ae9a42d797f2f0c913534ba0` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:48.143588 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81cceb0-6e01-4543-b6df-18a6b2deba11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73734ac1ae9a42d797f2f0c913534ba0)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:48.145572 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81cceb0-6e01-4543-b6df-18a6b2deba11 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:48.213844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7aa50-468f-44be-ab3c-6c730ece36ff admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:48.215051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-e0c7aa50-468f-44be-ab3c-6c730ece36ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:48.236804 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7aa50-468f-44be-ab3c-6c730ece36ff admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:48.248199 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20b150-4678-4571-a66d-28b3ee11f097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:48.248883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20b150-4678-4571-a66d-28b3ee11f097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:48.249187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20b150-4678-4571-a66d-28b3ee11f097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:48.249448 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20b150-4678-4571-a66d-28b3ee11f097 admin admin] PATH_INFO: `/v3/projects/35f34cd06c8e49f68ceac844d2cc528f` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:48.251316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20b150-4678-4571-a66d-28b3ee11f097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35f34cd06c8e49f68ceac844d2cc528f)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:48.253556 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20b150-4678-4571-a66d-28b3ee11f097 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:48.354616 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d81cceb0-6e01-4543-b6df-18a6b2deba11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73734ac1ae9a42d797f2f0c913534ba0'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.370153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6e20b150-4678-4571-a66d-28b3ee11f097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35f34cd06c8e49f68ceac844d2cc528f', 'request_id': 'req-6e20b150-4678-4571-a66d-28b3ee11f097'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.376498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6e20b150-4678-4571-a66d-28b3ee11f097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35f34cd06c8e49f68ceac844d2cc528f', 'request_id': 'req-6e20b150-4678-4571-a66d-28b3ee11f097'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.383131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6e20b150-4678-4571-a66d-28b3ee11f097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35f34cd06c8e49f68ceac844d2cc528f', 'request_id': 'req-6e20b150-4678-4571-a66d-28b3ee11f097'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.417266 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d81cceb0-6e01-4543-b6df-18a6b2deba11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73734ac1ae9a42d797f2f0c913534ba0'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.417945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d81cceb0-6e01-4543-b6df-18a6b2deba11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73734ac1ae9a42d797f2f0c913534ba0'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.426209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d81cceb0-6e01-4543-b6df-18a6b2deba11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73734ac1ae9a42d797f2f0c913534ba0'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.438069 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d81cceb0-6e01-4543-b6df-18a6b2deba11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73734ac1ae9a42d797f2f0c913534ba0'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.444689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 971/1910] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:48 2022] DELETE /identity/v3/users/73734ac1ae9a42d797f2f0c913534ba0 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:48.483860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6e20b150-4678-4571-a66d-28b3ee11f097 admin admin] The token cache is being invalidate because project 35f34cd06c8e49f68ceac844d2cc528f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:48.486042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6e20b150-4678-4571-a66d-28b3ee11f097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.486808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6e20b150-4678-4571-a66d-28b3ee11f097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.489508 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 940/1911] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:48 2022] DELETE /identity/v3/projects/35f34cd06c8e49f68ceac844d2cc528f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:48.493982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cceb0-6e01-4543-b6df-18a6b2deba11 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:48.495484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-d81cceb0-6e01-4543-b6df-18a6b2deba11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:48.564916 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cceb0-6e01-4543-b6df-18a6b2deba11 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:48.569540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f5eac-0576-4658-9a2e-157614b0d450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:48.570168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f5eac-0576-4658-9a2e-157614b0d450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:48.570446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f5eac-0576-4658-9a2e-157614b0d450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:48.570714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f5eac-0576-4658-9a2e-157614b0d450 admin admin] PATH_INFO: `/v3/users/312c439c4881416c8fdc381c7b322178` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:48.571843 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f5eac-0576-4658-9a2e-157614b0d450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=312c439c4881416c8fdc381c7b322178)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:48.573769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f5eac-0576-4658-9a2e-157614b0d450 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:48.810959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:48.817934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d70f5eac-0576-4658-9a2e-157614b0d450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '312c439c4881416c8fdc381c7b322178'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.885569 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d70f5eac-0576-4658-9a2e-157614b0d450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '312c439c4881416c8fdc381c7b322178'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.886179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d70f5eac-0576-4658-9a2e-157614b0d450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '312c439c4881416c8fdc381c7b322178'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.894072 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d70f5eac-0576-4658-9a2e-157614b0d450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '312c439c4881416c8fdc381c7b322178'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.906049 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d70f5eac-0576-4658-9a2e-157614b0d450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '312c439c4881416c8fdc381c7b322178'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:48.912980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 972/1912] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:48 2022] DELETE /identity/v3/users/312c439c4881416c8fdc381c7b322178 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:48.928102 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20b150-4678-4571-a66d-28b3ee11f097 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:48.929270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6e20b150-4678-4571-a66d-28b3ee11f097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:49.008395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20b150-4678-4571-a66d-28b3ee11f097 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:49.064465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84502e25-9717-41af-ab17-27d0a25aabcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:49.065112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84502e25-9717-41af-ab17-27d0a25aabcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:49.065393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84502e25-9717-41af-ab17-27d0a25aabcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:49.065664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84502e25-9717-41af-ab17-27d0a25aabcc admin admin] PATH_INFO: `/v3/projects/91e0223835e344aea8c5c266ae012a08` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:49.066964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84502e25-9717-41af-ab17-27d0a25aabcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=91e0223835e344aea8c5c266ae012a08)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:49.069024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84502e25-9717-41af-ab17-27d0a25aabcc admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:49.075412 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:49.191264 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-84502e25-9717-41af-ab17-27d0a25aabcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e0223835e344aea8c5c266ae012a08', 'request_id': 'req-84502e25-9717-41af-ab17-27d0a25aabcc'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:49.193971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-84502e25-9717-41af-ab17-27d0a25aabcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e0223835e344aea8c5c266ae012a08', 'request_id': 'req-84502e25-9717-41af-ab17-27d0a25aabcc'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:49.201780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-84502e25-9717-41af-ab17-27d0a25aabcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e0223835e344aea8c5c266ae012a08', 'request_id': 'req-84502e25-9717-41af-ab17-27d0a25aabcc'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:49.283594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-84502e25-9717-41af-ab17-27d0a25aabcc admin admin] The token cache is being invalidate because project 91e0223835e344aea8c5c266ae012a08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:49.284610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-84502e25-9717-41af-ab17-27d0a25aabcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:49.285158 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-84502e25-9717-41af-ab17-27d0a25aabcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:49.286762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 941/1913] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:48 2022] DELETE /identity/v3/projects/91e0223835e344aea8c5c266ae012a08 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:49.292988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05dc43-097b-4a09-b146-06d1c2c6a1ae None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:49.293441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05dc43-097b-4a09-b146-06d1c2c6a1ae None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:49.293573 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05dc43-097b-4a09-b146-06d1c2c6a1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:49.313645 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-8a05dc43-097b-4a09-b146-06d1c2c6a1ae None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:49.314872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-8a05dc43-097b-4a09-b146-06d1c2c6a1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:49.374695 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84502e25-9717-41af-ab17-27d0a25aabcc admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:49.374695 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-84502e25-9717-41af-ab17-27d0a25aabcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:49.389235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 973/1914] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:49.399714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878f339-8fec-49ce-b6c1-6924020cdea0 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:49.401113 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878f339-8fec-49ce-b6c1-6924020cdea0 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:49.401758 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878f339-8fec-49ce-b6c1-6924020cdea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:49.428013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-7878f339-8fec-49ce-b6c1-6924020cdea0 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:49.429596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7878f339-8fec-49ce-b6c1-6924020cdea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:49.440384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 974/1915] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:49.445112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878f339-8fec-49ce-b6c1-6924020cdea0 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:49.456779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878f339-8fec-49ce-b6c1-6924020cdea0 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:49.462619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da333545-0b99-4642-a639-34241e8bbc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:49.463462 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da333545-0b99-4642-a639-34241e8bbc40 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:49.463858 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da333545-0b99-4642-a639-34241e8bbc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:49.464207 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da333545-0b99-4642-a639-34241e8bbc40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:49.465915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da333545-0b99-4642-a639-34241e8bbc40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:19:49.465915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da333545-0b99-4642-a639-34241e8bbc40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:49.467267 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da333545-0b99-4642-a639-34241e8bbc40 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:49.495856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 975/1916] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:19:49.501071 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da333545-0b99-4642-a639-34241e8bbc40 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:49.504048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84502e25-9717-41af-ab17-27d0a25aabcc admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:49.509885 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f282f-da74-4ae0-a1ef-db8bd346638c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:49.510140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f282f-da74-4ae0-a1ef-db8bd346638c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:49.510490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f282f-da74-4ae0-a1ef-db8bd346638c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:49.510963 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f282f-da74-4ae0-a1ef-db8bd346638c admin admin] PATH_INFO: `/v3/projects/5b7fb8d73d984ce284490d2cf092f479` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:49.515872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f282f-da74-4ae0-a1ef-db8bd346638c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7fb8d73d984ce284490d2cf092f479)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:49.515985 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da333545-0b99-4642-a639-34241e8bbc40 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:49.516820 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f282f-da74-4ae0-a1ef-db8bd346638c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:49.519657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd639a-3980-454f-8fc1-04f595df83d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:49.520462 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd639a-3980-454f-8fc1-04f595df83d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:49.521073 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd639a-3980-454f-8fc1-04f595df83d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:49.521607 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd639a-3980-454f-8fc1-04f595df83d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:49.522377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd639a-3980-454f-8fc1-04f595df83d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:49.524815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd639a-3980-454f-8fc1-04f595df83d0 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:49.650987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1d0f282f-da74-4ae0-a1ef-db8bd346638c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7fb8d73d984ce284490d2cf092f479', 'request_id': 'req-1d0f282f-da74-4ae0-a1ef-db8bd346638c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:49.656416 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1d0f282f-da74-4ae0-a1ef-db8bd346638c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7fb8d73d984ce284490d2cf092f479', 'request_id': 'req-1d0f282f-da74-4ae0-a1ef-db8bd346638c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:49.663070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1d0f282f-da74-4ae0-a1ef-db8bd346638c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7fb8d73d984ce284490d2cf092f479', 'request_id': 'req-1d0f282f-da74-4ae0-a1ef-db8bd346638c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:49.673461 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 976/1917] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:49 2022] POST /identity/v3/projects => generated 359 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:49.688923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd639a-3980-454f-8fc1-04f595df83d0 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:49.741291 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd639a-3980-454f-8fc1-04f595df83d0 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:49.745279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1d0f282f-da74-4ae0-a1ef-db8bd346638c admin admin] The token cache is being invalidate because project 5b7fb8d73d984ce284490d2cf092f479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:49.745579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1d0f282f-da74-4ae0-a1ef-db8bd346638c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:49.746215 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1d0f282f-da74-4ae0-a1ef-db8bd346638c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:49.747714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 942/1918] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:49 2022] DELETE /identity/v3/projects/5b7fb8d73d984ce284490d2cf092f479 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:49.755648 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2392883-16c7-4568-9d27-9bf2f6e5d78f None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:49.755783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca90e2-5a6f-4bc4-959f-f420910dfaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:49.755990 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2392883-16c7-4568-9d27-9bf2f6e5d78f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:49.756252 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2392883-16c7-4568-9d27-9bf2f6e5d78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:49.756353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca90e2-5a6f-4bc4-959f-f420910dfaeb admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:49.756625 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca90e2-5a6f-4bc4-959f-f420910dfaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:49.756950 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca90e2-5a6f-4bc4-959f-f420910dfaeb admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:49.757598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca90e2-5a6f-4bc4-959f-f420910dfaeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:49.759548 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca90e2-5a6f-4bc4-959f-f420910dfaeb admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:49.775991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-b2392883-16c7-4568-9d27-9bf2f6e5d78f None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:49.777227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b2392883-16c7-4568-9d27-9bf2f6e5d78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:49.785749 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 943/1919] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:49.794339 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0c530-d2b6-410a-8a0c-84492201972c None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:49.794625 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0c530-d2b6-410a-8a0c-84492201972c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:49.794902 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0c530-d2b6-410a-8a0c-84492201972c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:49.819817 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-cbf0c530-d2b6-410a-8a0c-84492201972c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:49.821066 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-cbf0c530-d2b6-410a-8a0c-84492201972c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:49.824790 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 977/1920] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:49 2022] POST /identity/v3/users => generated 280 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:19:49.828805 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca90e2-5a6f-4bc4-959f-f420910dfaeb admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:49.830261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 944/1921] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:49.830738 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b1ca90e2-5a6f-4bc4-959f-f420910dfaeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:49.834435 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0c530-d2b6-410a-8a0c-84492201972c None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:49.839424 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:49.842847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca90e2-5a6f-4bc4-959f-f420910dfaeb admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:49.844666 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0c530-d2b6-410a-8a0c-84492201972c None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:49.847989 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01dd9b-c8bc-48b1-9851-635c4d5ceee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:49.848577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01dd9b-c8bc-48b1-9851-635c4d5ceee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:49.848909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01dd9b-c8bc-48b1-9851-635c4d5ceee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:49.849183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01dd9b-c8bc-48b1-9851-635c4d5ceee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:49.849553 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7dfccc-29f7-4b16-a74d-0b0e56f87f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:49.849823 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01dd9b-c8bc-48b1-9851-635c4d5ceee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:49.850149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dfccc-29f7-4b16-a74d-0b0e56f87f66 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:49.850429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dfccc-29f7-4b16-a74d-0b0e56f87f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:49.850706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dfccc-29f7-4b16-a74d-0b0e56f87f66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:49.851351 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7dfccc-29f7-4b16-a74d-0b0e56f87f66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:19:49.851445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01dd9b-c8bc-48b1-9851-635c4d5ceee1 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:49.851656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7dfccc-29f7-4b16-a74d-0b0e56f87f66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:49.853567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7dfccc-29f7-4b16-a74d-0b0e56f87f66 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:49.860596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:49.861955 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 978/1922] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:19:49.868181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 945/1923] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:19:49.870913 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01dd9b-c8bc-48b1-9851-635c4d5ceee1 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:49.871846 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7dfccc-29f7-4b16-a74d-0b0e56f87f66 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:49.881389 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01dd9b-c8bc-48b1-9851-635c4d5ceee1 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:49.882262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7dfccc-29f7-4b16-a74d-0b0e56f87f66 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:49.886120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432de2bd-7150-4e02-8c64-9dc6b8dba6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:49.886710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432de2bd-7150-4e02-8c64-9dc6b8dba6b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:49.886859 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a1bd7-7c4e-4314-8831-be8fb5566921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:49.887002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432de2bd-7150-4e02-8c64-9dc6b8dba6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:49.887278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432de2bd-7150-4e02-8c64-9dc6b8dba6b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:49.887438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a1bd7-7c4e-4314-8831-be8fb5566921 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:49.887717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a1bd7-7c4e-4314-8831-be8fb5566921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:49.887924 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432de2bd-7150-4e02-8c64-9dc6b8dba6b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:49.888063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a1bd7-7c4e-4314-8831-be8fb5566921 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:49.888592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a1bd7-7c4e-4314-8831-be8fb5566921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:49.889612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432de2bd-7150-4e02-8c64-9dc6b8dba6b3 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:49.890479 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a1bd7-7c4e-4314-8831-be8fb5566921 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:49.900612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 979/1924] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:19:49.905556 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432de2bd-7150-4e02-8c64-9dc6b8dba6b3 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:49.915821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432de2bd-7150-4e02-8c64-9dc6b8dba6b3 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:49.920436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bb89c-1a8c-4b9a-bb82-debb17794e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:49.921122 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bb89c-1a8c-4b9a-bb82-debb17794e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:49.921409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bb89c-1a8c-4b9a-bb82-debb17794e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:49.921691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bb89c-1a8c-4b9a-bb82-debb17794e93 admin admin] PATH_INFO: `/v3/projects/17746eb8c4684fb18c57c3dd0d2e3264/users/239fccce8ab541c4b7e53a1951a391eb/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:49.940870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bb89c-1a8c-4b9a-bb82-debb17794e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17746eb8c4684fb18c57c3dd0d2e3264, user_id=239fccce8ab541c4b7e53a1951a391eb, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:49.940870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bb89c-1a8c-4b9a-bb82-debb17794e93 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:50.093169 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 946/1925] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:49 2022] POST /identity/v3/projects => generated 383 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:19:50.097332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a1bd7-7c4e-4314-8831-be8fb5566921 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:50.140585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 980/1926] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:49 2022] PUT /identity/v3/projects/17746eb8c4684fb18c57c3dd0d2e3264/users/239fccce8ab541c4b7e53a1951a391eb/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:19:50.155138 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27487-0854-41f1-96a3-e45d58f2f7a8 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:50.155702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27487-0854-41f1-96a3-e45d58f2f7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:50.156079 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27487-0854-41f1-96a3-e45d58f2f7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:50.166002 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a1bd7-7c4e-4314-8831-be8fb5566921 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:50.186979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-c9c27487-0854-41f1-96a3-e45d58f2f7a8 None None] MFA Rules not processed for user `239fccce8ab541c4b7e53a1951a391eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:50.188304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-c9c27487-0854-41f1-96a3-e45d58f2f7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:50.229806 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ebd1e-35a6-4c0c-a5ae-298395231064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:50.230400 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ebd1e-35a6-4c0c-a5ae-298395231064 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:50.230678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ebd1e-35a6-4c0c-a5ae-298395231064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:50.230950 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ebd1e-35a6-4c0c-a5ae-298395231064 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:50.231576 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ebd1e-35a6-4c0c-a5ae-298395231064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:50.233525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ebd1e-35a6-4c0c-a5ae-298395231064 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:50.255310 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 981/1927] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:50 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:50.290900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c27487-0854-41f1-96a3-e45d58f2f7a8 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:50.292369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-c9c27487-0854-41f1-96a3-e45d58f2f7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:50.330022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 947/1928] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:50 2022] POST /identity/v3/users => generated 292 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:19:50.334025 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ebd1e-35a6-4c0c-a5ae-298395231064 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:50.343931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ebd1e-35a6-4c0c-a5ae-298395231064 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:50.348820 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88685a-187d-4264-aaf8-a4fdf86d2a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:50.350197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88685a-187d-4264-aaf8-a4fdf86d2a61 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:50.350611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88685a-187d-4264-aaf8-a4fdf86d2a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:50.351079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88685a-187d-4264-aaf8-a4fdf86d2a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:50.351739 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88685a-187d-4264-aaf8-a4fdf86d2a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:50.352066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c27487-0854-41f1-96a3-e45d58f2f7a8 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:50.353421 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88685a-187d-4264-aaf8-a4fdf86d2a61 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:50.355986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92315509-cac3-4724-a1b7-54e7300bbbec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:50.357029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92315509-cac3-4724-a1b7-54e7300bbbec service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:50.357309 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92315509-cac3-4724-a1b7-54e7300bbbec service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:50.357586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92315509-cac3-4724-a1b7-54e7300bbbec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:50.365264 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 948/1929] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:19:50.371433 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88685a-187d-4264-aaf8-a4fdf86d2a61 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:50.371578 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92315509-cac3-4724-a1b7-54e7300bbbec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:50.373239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92315509-cac3-4724-a1b7-54e7300bbbec service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:50.379975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88685a-187d-4264-aaf8-a4fdf86d2a61 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:50.385360 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 982/1930] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:50 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:19:50.390303 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980c2c1-32f9-4ae7-9f74-206f74cea3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:50.390863 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980c2c1-32f9-4ae7-9f74-206f74cea3ee admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:50.391819 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980c2c1-32f9-4ae7-9f74-206f74cea3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:50.392219 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980c2c1-32f9-4ae7-9f74-206f74cea3ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:50.392895 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980c2c1-32f9-4ae7-9f74-206f74cea3ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:50.394482 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980c2c1-32f9-4ae7-9f74-206f74cea3ee admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:50.409600 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92315509-cac3-4724-a1b7-54e7300bbbec service cinder] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:50.428956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92315509-cac3-4724-a1b7-54e7300bbbec service cinder] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:50.429166 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 949/1931] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:50.430126 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980c2c1-32f9-4ae7-9f74-206f74cea3ee admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:50.440791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4d8c4-c286-4903-a626-c091a97573b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:50.440791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4d8c4-c286-4903-a626-c091a97573b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:50.440791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4d8c4-c286-4903-a626-c091a97573b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:50.440791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4d8c4-c286-4903-a626-c091a97573b1 admin admin] PATH_INFO: `/v3/users/239fccce8ab541c4b7e53a1951a391eb` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:50.444148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4d8c4-c286-4903-a626-c091a97573b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=239fccce8ab541c4b7e53a1951a391eb)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:50.446364 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4d8c4-c286-4903-a626-c091a97573b1 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:50.452721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980c2c1-32f9-4ae7-9f74-206f74cea3ee admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:50.467464 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5fe33-5c57-40b7-9e3f-9dd6a6d86b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:50.468240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5fe33-5c57-40b7-9e3f-9dd6a6d86b42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:50.468527 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5fe33-5c57-40b7-9e3f-9dd6a6d86b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:50.468848 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5fe33-5c57-40b7-9e3f-9dd6a6d86b42 admin admin] PATH_INFO: `/v3/projects/8a52eb0debe5478b976b1d3193a872e2/users/76470494a470481ba07bcb261a3b68e4/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:50.489151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5fe33-5c57-40b7-9e3f-9dd6a6d86b42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a52eb0debe5478b976b1d3193a872e2, user_id=76470494a470481ba07bcb261a3b68e4, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:50.491872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5fe33-5c57-40b7-9e3f-9dd6a6d86b42 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:50.589356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 950/1932] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:50 2022] PUT /identity/v3/projects/8a52eb0debe5478b976b1d3193a872e2/users/76470494a470481ba07bcb261a3b68e4/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:50.601875 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c00b3-ca27-40af-98a2-b38d673e41c2 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:50.602164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c00b3-ca27-40af-98a2-b38d673e41c2 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:50.602480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c00b3-ca27-40af-98a2-b38d673e41c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:50.633125 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-1f8c00b3-ca27-40af-98a2-b38d673e41c2 None None] MFA Rules not processed for user `76470494a470481ba07bcb261a3b68e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:50.634409 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1f8c00b3-ca27-40af-98a2-b38d673e41c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:50.648050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:50.649325 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:50.650900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7da4d8c4-c286-4903-a626-c091a97573b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '239fccce8ab541c4b7e53a1951a391eb'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:50.720934 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 951/1933] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:50.721966 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c00b3-ca27-40af-98a2-b38d673e41c2 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:50.726614 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7da4d8c4-c286-4903-a626-c091a97573b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '239fccce8ab541c4b7e53a1951a391eb'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:50.727290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7da4d8c4-c286-4903-a626-c091a97573b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '239fccce8ab541c4b7e53a1951a391eb'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:50.737910 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7da4d8c4-c286-4903-a626-c091a97573b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '239fccce8ab541c4b7e53a1951a391eb'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:50.754498 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-7da4d8c4-c286-4903-a626-c091a97573b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '239fccce8ab541c4b7e53a1951a391eb'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:50.765023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 983/1934] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:50 2022] DELETE /identity/v3/users/239fccce8ab541c4b7e53a1951a391eb => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:50.767230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ae384-67a5-4a09-afc1-7195223d9ffc None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:50.771943 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ae384-67a5-4a09-afc1-7195223d9ffc None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:50.771943 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ae384-67a5-4a09-afc1-7195223d9ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:50.795276 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-da2ae384-67a5-4a09-afc1-7195223d9ffc None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:50.795276 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-da2ae384-67a5-4a09-afc1-7195223d9ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:50.795531 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c00b3-ca27-40af-98a2-b38d673e41c2 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:50.857142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d08aa-274a-4680-b80a-757dc91e6d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:50.857750 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d08aa-274a-4680-b80a-757dc91e6d05 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:50.858033 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d08aa-274a-4680-b80a-757dc91e6d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:50.858554 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d08aa-274a-4680-b80a-757dc91e6d05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:50.859186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d08aa-274a-4680-b80a-757dc91e6d05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:50.861413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d08aa-274a-4680-b80a-757dc91e6d05 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:50.880141 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 984/1935] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:50.895055 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ae384-67a5-4a09-afc1-7195223d9ffc None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:50.897397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-da2ae384-67a5-4a09-afc1-7195223d9ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:50.964074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ae384-67a5-4a09-afc1-7195223d9ffc None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:50.969921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3546c-52b1-4ffa-bc93-b9fb6732692f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:50.971104 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3546c-52b1-4ffa-bc93-b9fb6732692f service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:50.971104 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3546c-52b1-4ffa-bc93-b9fb6732692f service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:50.971445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3546c-52b1-4ffa-bc93-b9fb6732692f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:50.974727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 952/1936] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:50 2022] POST /identity/v3/projects => generated 383 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:50.982067 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3546c-52b1-4ffa-bc93-b9fb6732692f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:50.983775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3546c-52b1-4ffa-bc93-b9fb6732692f service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:50.985774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d08aa-274a-4680-b80a-757dc91e6d05 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:50.987516 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2c5d08aa-274a-4680-b80a-757dc91e6d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:51.013243 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 985/1937] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:51.048945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:51.065623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d08aa-274a-4680-b80a-757dc91e6d05 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:51.070432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb887568-3f44-4d04-96f0-83cbd57b5641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:51.071037 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb887568-3f44-4d04-96f0-83cbd57b5641 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:51.071319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb887568-3f44-4d04-96f0-83cbd57b5641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:51.071591 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb887568-3f44-4d04-96f0-83cbd57b5641 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:51.072217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb887568-3f44-4d04-96f0-83cbd57b5641 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:51.074236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb887568-3f44-4d04-96f0-83cbd57b5641 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:51.169344 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 953/1938] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:50 2022] POST /identity/v3/users => generated 298 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:51.173563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3546c-52b1-4ffa-bc93-b9fb6732692f service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:51.183645 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3546c-52b1-4ffa-bc93-b9fb6732692f service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:51.189955 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09555ebf-db69-47c3-b3f5-78fa82ee5105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:51.189955 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09555ebf-db69-47c3-b3f5-78fa82ee5105 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:51.189955 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09555ebf-db69-47c3-b3f5-78fa82ee5105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:51.189955 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09555ebf-db69-47c3-b3f5-78fa82ee5105 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:51.190411 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09555ebf-db69-47c3-b3f5-78fa82ee5105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:51.191994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09555ebf-db69-47c3-b3f5-78fa82ee5105 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:51.205852 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 986/1939] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:19:51.207089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb887568-3f44-4d04-96f0-83cbd57b5641 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:51.216999 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb887568-3f44-4d04-96f0-83cbd57b5641 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:51.222408 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8996e-d127-4309-a395-2233790ebf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:51.223063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8996e-d127-4309-a395-2233790ebf3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:51.223343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8996e-d127-4309-a395-2233790ebf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:51.223619 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8996e-d127-4309-a395-2233790ebf3c admin admin] PATH_INFO: `/v3/projects/1316854811d04a94a359e7b103bc168c/users/c02bbe18bc7e4f0daa17c9f8e5d02108/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:51.246220 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8996e-d127-4309-a395-2233790ebf3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1316854811d04a94a359e7b103bc168c, user_id=c02bbe18bc7e4f0daa17c9f8e5d02108, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:51.248965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8996e-d127-4309-a395-2233790ebf3c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:51.308862 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 954/1940] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:51 2022] PUT /identity/v3/projects/1316854811d04a94a359e7b103bc168c/users/c02bbe18bc7e4f0daa17c9f8e5d02108/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:51.313300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09555ebf-db69-47c3-b3f5-78fa82ee5105 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:51.323731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09555ebf-db69-47c3-b3f5-78fa82ee5105 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:51.375549 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:51.382466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4355ab4-26f1-4f8a-9399-9822217b2790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:51.383071 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4355ab4-26f1-4f8a-9399-9822217b2790 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:51.383352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4355ab4-26f1-4f8a-9399-9822217b2790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:51.383629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4355ab4-26f1-4f8a-9399-9822217b2790 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:51.384272 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4355ab4-26f1-4f8a-9399-9822217b2790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:51.385913 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4355ab4-26f1-4f8a-9399-9822217b2790 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:51.401403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 987/1941] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:51.402586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8996e-d127-4309-a395-2233790ebf3c admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:51.413652 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8996e-d127-4309-a395-2233790ebf3c admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:51.418593 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f138ba8-8be3-4166-889a-d59bb604f871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:51.419171 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f138ba8-8be3-4166-889a-d59bb604f871 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:51.419457 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f138ba8-8be3-4166-889a-d59bb604f871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:51.419731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f138ba8-8be3-4166-889a-d59bb604f871 admin admin] PATH_INFO: `/v3/projects/1316854811d04a94a359e7b103bc168c/users/c02bbe18bc7e4f0daa17c9f8e5d02108/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:51.421637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f138ba8-8be3-4166-889a-d59bb604f871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1316854811d04a94a359e7b103bc168c, user_id=c02bbe18bc7e4f0daa17c9f8e5d02108, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:51.424475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f138ba8-8be3-4166-889a-d59bb604f871 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:51.435399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 955/1942] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:51 2022] PUT /identity/v3/projects/1316854811d04a94a359e7b103bc168c/users/c02bbe18bc7e4f0daa17c9f8e5d02108/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:51.446720 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d17697-f806-40af-b71f-479add199342 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:51.447010 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d17697-f806-40af-b71f-479add199342 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:51.447366 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d17697-f806-40af-b71f-479add199342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:51.467440 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f138ba8-8be3-4166-889a-d59bb604f871 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:51.470246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4f138ba8-8be3-4166-889a-d59bb604f871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:51.478901 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-e7d17697-f806-40af-b71f-479add199342 None None] MFA Rules not processed for user `c02bbe18bc7e4f0daa17c9f8e5d02108`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:51.480534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e7d17697-f806-40af-b71f-479add199342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:51.551223 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f138ba8-8be3-4166-889a-d59bb604f871 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:51.555105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 988/1943] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:51.555917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58d4c1-32dd-4b28-9ca9-7f642dae4524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:51.556538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58d4c1-32dd-4b28-9ca9-7f642dae4524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:51.556849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58d4c1-32dd-4b28-9ca9-7f642dae4524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:51.557125 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58d4c1-32dd-4b28-9ca9-7f642dae4524 admin admin] PATH_INFO: `/v3/projects/17746eb8c4684fb18c57c3dd0d2e3264` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:51.558280 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58d4c1-32dd-4b28-9ca9-7f642dae4524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17746eb8c4684fb18c57c3dd0d2e3264)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:51.560247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58d4c1-32dd-4b28-9ca9-7f642dae4524 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:51.567239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d17697-f806-40af-b71f-479add199342 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:51.568619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e7d17697-f806-40af-b71f-479add199342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:51.624808 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d17697-f806-40af-b71f-479add199342 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:51.629719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bfb602-5aa6-4390-af76-7d27052da52b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:51.630349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfb602-5aa6-4390-af76-7d27052da52b service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:51.630630 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfb602-5aa6-4390-af76-7d27052da52b service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:51.630901 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfb602-5aa6-4390-af76-7d27052da52b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:51.633274 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-34bfb602-5aa6-4390-af76-7d27052da52b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:51.702506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bfb602-5aa6-4390-af76-7d27052da52b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:51.704156 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-bb58d4c1-32dd-4b28-9ca9-7f642dae4524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17746eb8c4684fb18c57c3dd0d2e3264', 'request_id': 'req-bb58d4c1-32dd-4b28-9ca9-7f642dae4524'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:51.705516 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bfb602-5aa6-4390-af76-7d27052da52b service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:51.710155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-bb58d4c1-32dd-4b28-9ca9-7f642dae4524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17746eb8c4684fb18c57c3dd0d2e3264', 'request_id': 'req-bb58d4c1-32dd-4b28-9ca9-7f642dae4524'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:51.717099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-bb58d4c1-32dd-4b28-9ca9-7f642dae4524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17746eb8c4684fb18c57c3dd0d2e3264', 'request_id': 'req-bb58d4c1-32dd-4b28-9ca9-7f642dae4524'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:51.718672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 989/1944] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:19:51.795093 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-bb58d4c1-32dd-4b28-9ca9-7f642dae4524 admin admin] The token cache is being invalidate because project 17746eb8c4684fb18c57c3dd0d2e3264 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:51.795390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-bb58d4c1-32dd-4b28-9ca9-7f642dae4524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:51.796196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-bb58d4c1-32dd-4b28-9ca9-7f642dae4524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:51.798145 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 956/1945] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:51 2022] DELETE /identity/v3/projects/17746eb8c4684fb18c57c3dd0d2e3264 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:51.805118 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397412a-e3c3-4054-acb3-5be16c316fb3 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:51.805479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397412a-e3c3-4054-acb3-5be16c316fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:51.805822 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397412a-e3c3-4054-acb3-5be16c316fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:51.829277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-7397412a-e3c3-4054-acb3-5be16c316fb3 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:51.830635 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7397412a-e3c3-4054-acb3-5be16c316fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:51.878627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:51.919080 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 990/1946] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:51.927981 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1bc5c-4c94-4bbc-9af7-379feede8b7c None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:51.932723 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1bc5c-4c94-4bbc-9af7-379feede8b7c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:51.932723 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1bc5c-4c94-4bbc-9af7-379feede8b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:51.948460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-73a1bc5c-4c94-4bbc-9af7-379feede8b7c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:51.949706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-73a1bc5c-4c94-4bbc-9af7-379feede8b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:51.958160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 957/1947] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:51.962465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397412a-e3c3-4054-acb3-5be16c316fb3 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:51.973150 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397412a-e3c3-4054-acb3-5be16c316fb3 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:51.977998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e4332-88cb-4837-b688-ab99a4a495ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:51.978606 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e4332-88cb-4837-b688-ab99a4a495ea admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:51.978886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e4332-88cb-4837-b688-ab99a4a495ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:51.979160 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e4332-88cb-4837-b688-ab99a4a495ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:51.979808 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e4332-88cb-4837-b688-ab99a4a495ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:19:51.980171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e4332-88cb-4837-b688-ab99a4a495ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:51.981901 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e4332-88cb-4837-b688-ab99a4a495ea admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:51.994827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 991/1948] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:19:52.001643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1bc5c-4c94-4bbc-9af7-379feede8b7c None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:52.011272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1bc5c-4c94-4bbc-9af7-379feede8b7c None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:52.035787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42d091-b686-4941-ac7c-2ff44d4a1833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:52.036420 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42d091-b686-4941-ac7c-2ff44d4a1833 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:52.036724 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42d091-b686-4941-ac7c-2ff44d4a1833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:52.036997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42d091-b686-4941-ac7c-2ff44d4a1833 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:52.037676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42d091-b686-4941-ac7c-2ff44d4a1833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:52.039988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42d091-b686-4941-ac7c-2ff44d4a1833 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:52.149732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 958/1949] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:51 2022] POST /identity/v3/projects => generated 369 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:19:52.151225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e4332-88cb-4837-b688-ab99a4a495ea admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:52.164079 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e4332-88cb-4837-b688-ab99a4a495ea admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:52.233499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cedad-05e6-404b-a91b-34d7137e60e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:52.234424 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cedad-05e6-404b-a91b-34d7137e60e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:52.234748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cedad-05e6-404b-a91b-34d7137e60e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:52.235023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cedad-05e6-404b-a91b-34d7137e60e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:52.235652 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cedad-05e6-404b-a91b-34d7137e60e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:52.238175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cedad-05e6-404b-a91b-34d7137e60e4 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:52.342620 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 992/1950] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:52 2022] POST /identity/v3/users => generated 285 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:19:52.346910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42d091-b686-4941-ac7c-2ff44d4a1833 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:52.352289 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:52.357529 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42d091-b686-4941-ac7c-2ff44d4a1833 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:52.362189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17efee2-04fa-4a4d-b1da-3ec85b521c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:52.362776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17efee2-04fa-4a4d-b1da-3ec85b521c8b admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:52.363054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17efee2-04fa-4a4d-b1da-3ec85b521c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:52.363324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17efee2-04fa-4a4d-b1da-3ec85b521c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:52.363961 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17efee2-04fa-4a4d-b1da-3ec85b521c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:52.365556 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17efee2-04fa-4a4d-b1da-3ec85b521c8b admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:52.375702 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 959/1951] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:52.380004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cedad-05e6-404b-a91b-34d7137e60e4 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:52.389930 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cedad-05e6-404b-a91b-34d7137e60e4 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:52.395101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6616021-c1ef-4f0d-8e9f-18239d557f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:52.395688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6616021-c1ef-4f0d-8e9f-18239d557f33 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:52.395970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6616021-c1ef-4f0d-8e9f-18239d557f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:52.396241 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6616021-c1ef-4f0d-8e9f-18239d557f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:52.396912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6616021-c1ef-4f0d-8e9f-18239d557f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:52.398527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6616021-c1ef-4f0d-8e9f-18239d557f33 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:52.408260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 993/1952] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:52.412454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17efee2-04fa-4a4d-b1da-3ec85b521c8b admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:52.422987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17efee2-04fa-4a4d-b1da-3ec85b521c8b admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:52.427492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e10fc1-f152-4904-b8f5-10fbbff3fec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:52.428132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e10fc1-f152-4904-b8f5-10fbbff3fec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:52.428416 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e10fc1-f152-4904-b8f5-10fbbff3fec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:52.428778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e10fc1-f152-4904-b8f5-10fbbff3fec6 admin admin] PATH_INFO: `/v3/projects/8e68b526cb844c619138688fc68e459f/users/f48af9d33a614ab4b100cd72848c7c30/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:52.445046 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e10fc1-f152-4904-b8f5-10fbbff3fec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e68b526cb844c619138688fc68e459f, user_id=f48af9d33a614ab4b100cd72848c7c30, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:52.456672 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e10fc1-f152-4904-b8f5-10fbbff3fec6 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:52.555254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 960/1953] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:52 2022] PUT /identity/v3/projects/8e68b526cb844c619138688fc68e459f/users/f48af9d33a614ab4b100cd72848c7c30/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:52.566130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272f8d5-798b-47cf-a008-fb9dfecd672a None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:52.566440 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272f8d5-798b-47cf-a008-fb9dfecd672a None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:52.566710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272f8d5-798b-47cf-a008-fb9dfecd672a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:52.595937 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-2272f8d5-798b-47cf-a008-fb9dfecd672a None None] MFA Rules not processed for user `f48af9d33a614ab4b100cd72848c7c30`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:52.597090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2272f8d5-798b-47cf-a008-fb9dfecd672a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:52.660199 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 994/1954] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:52.673403 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e10fc1-f152-4904-b8f5-10fbbff3fec6 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:52.675185 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a7e10fc1-f152-4904-b8f5-10fbbff3fec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:52.754948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e10fc1-f152-4904-b8f5-10fbbff3fec6 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:52.758676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a1fb7-3ccb-4d8e-9b6c-574d46dd8b62 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe9f3fec7104a289e05e88e2d751458', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:52.759282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a1fb7-3ccb-4d8e-9b6c-574d46dd8b62 service glance] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:52.759566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a1fb7-3ccb-4d8e-9b6c-574d46dd8b62 service glance] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:52.759845 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a1fb7-3ccb-4d8e-9b6c-574d46dd8b62 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:52.768550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a1fb7-3ccb-4d8e-9b6c-574d46dd8b62 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:52.770010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a1fb7-3ccb-4d8e-9b6c-574d46dd8b62 service glance] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:52.778860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 961/1955] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:56.515190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272f8d5-798b-47cf-a008-fb9dfecd672a None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:56.517441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2272f8d5-798b-47cf-a008-fb9dfecd672a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:56.552024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:56.590165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272f8d5-798b-47cf-a008-fb9dfecd672a None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:56.595835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65fb69-20aa-4eb5-b872-1b39abd50cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:56.596463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65fb69-20aa-4eb5-b872-1b39abd50cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:56.596773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65fb69-20aa-4eb5-b872-1b39abd50cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:56.597034 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65fb69-20aa-4eb5-b872-1b39abd50cc4 admin admin] PATH_INFO: `/v3/users/f9c2a4e4989b42348010571c6103dc50` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:56.598154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65fb69-20aa-4eb5-b872-1b39abd50cc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9c2a4e4989b42348010571c6103dc50)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:56.600202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65fb69-20aa-4eb5-b872-1b39abd50cc4 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:56.821621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-da65fb69-20aa-4eb5-b872-1b39abd50cc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9c2a4e4989b42348010571c6103dc50'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:56.884719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-da65fb69-20aa-4eb5-b872-1b39abd50cc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9c2a4e4989b42348010571c6103dc50'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:56.885365 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-da65fb69-20aa-4eb5-b872-1b39abd50cc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9c2a4e4989b42348010571c6103dc50'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:56.894311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-da65fb69-20aa-4eb5-b872-1b39abd50cc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9c2a4e4989b42348010571c6103dc50'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:56.910010 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-da65fb69-20aa-4eb5-b872-1b39abd50cc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9c2a4e4989b42348010571c6103dc50'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:56.917193 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 995/1956] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:19:56 2022] DELETE /identity/v3/users/f9c2a4e4989b42348010571c6103dc50 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:56.921717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fe4fd-786d-4b38-8ae2-699533ea7a5b None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:56.922391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fe4fd-786d-4b38-8ae2-699533ea7a5b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:56.922391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fe4fd-786d-4b38-8ae2-699533ea7a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:56.944603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-4d8fe4fd-786d-4b38-8ae2-699533ea7a5b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:56.946187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4d8fe4fd-786d-4b38-8ae2-699533ea7a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:57.033745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 962/1957] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:57.046313 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65fb69-20aa-4eb5-b872-1b39abd50cc4 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:57.048016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-da65fb69-20aa-4eb5-b872-1b39abd50cc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:57.121606 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65fb69-20aa-4eb5-b872-1b39abd50cc4 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:57.126805 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278fff1-009d-45b5-befc-adac78ae3d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:57.128118 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278fff1-009d-45b5-befc-adac78ae3d8b service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:57.128118 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278fff1-009d-45b5-befc-adac78ae3d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:57.128118 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278fff1-009d-45b5-befc-adac78ae3d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:57.144597 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278fff1-009d-45b5-befc-adac78ae3d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:57.148018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278fff1-009d-45b5-befc-adac78ae3d8b service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:57.160438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 996/1958] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:19:57.699258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fe4fd-786d-4b38-8ae2-699533ea7a5b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:57.700856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4d8fe4fd-786d-4b38-8ae2-699533ea7a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:57.715694 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fe4fd-786d-4b38-8ae2-699533ea7a5b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:57.722933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bd2f4-351d-43ac-b18a-e10cd3c7625d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:57.723650 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bd2f4-351d-43ac-b18a-e10cd3c7625d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:57.737459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bd2f4-351d-43ac-b18a-e10cd3c7625d admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:57.738025 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bd2f4-351d-43ac-b18a-e10cd3c7625d admin admin] PATH_INFO: `/v3/projects/88f4251319c843aca548bf3546a12b2e` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:57.742123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bd2f4-351d-43ac-b18a-e10cd3c7625d admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f4251319c843aca548bf3546a12b2e)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:57.744861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bd2f4-351d-43ac-b18a-e10cd3c7625d admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:57.897737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:57.901020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-476bd2f4-351d-43ac-b18a-e10cd3c7625d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f4251319c843aca548bf3546a12b2e', 'request_id': 'req-476bd2f4-351d-43ac-b18a-e10cd3c7625d'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:57.907216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-476bd2f4-351d-43ac-b18a-e10cd3c7625d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f4251319c843aca548bf3546a12b2e', 'request_id': 'req-476bd2f4-351d-43ac-b18a-e10cd3c7625d'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:57.916855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-476bd2f4-351d-43ac-b18a-e10cd3c7625d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f4251319c843aca548bf3546a12b2e', 'request_id': 'req-476bd2f4-351d-43ac-b18a-e10cd3c7625d'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:58.034176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-476bd2f4-351d-43ac-b18a-e10cd3c7625d admin admin] The token cache is being invalidate because project 88f4251319c843aca548bf3546a12b2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:19:58.035620 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-476bd2f4-351d-43ac-b18a-e10cd3c7625d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:58.036455 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-476bd2f4-351d-43ac-b18a-e10cd3c7625d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:19:58.038211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 963/1959] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:19:57 2022] DELETE /identity/v3/projects/88f4251319c843aca548bf3546a12b2e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:58.043871 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740a61f-6515-4786-aec3-3d0a11592215 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:58.044353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740a61f-6515-4786-aec3-3d0a11592215 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:58.050368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740a61f-6515-4786-aec3-3d0a11592215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:58.074043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-d740a61f-6515-4786-aec3-3d0a11592215 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:58.074043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-d740a61f-6515-4786-aec3-3d0a11592215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:58.121477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:19:58.140950 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 997/1960] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:58.149180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf902d8-2e2e-47e5-beea-dc9d60277803 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:58.149837 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf902d8-2e2e-47e5-beea-dc9d60277803 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:58.150295 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf902d8-2e2e-47e5-beea-dc9d60277803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:58.171030 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-ebf902d8-2e2e-47e5-beea-dc9d60277803 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:58.173532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-ebf902d8-2e2e-47e5-beea-dc9d60277803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:58.188697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 964/1961] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:19:58.194442 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740a61f-6515-4786-aec3-3d0a11592215 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:58.199676 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740a61f-6515-4786-aec3-3d0a11592215 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:58.204991 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df8508-a04d-4107-9689-9f3eba082ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:58.205749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df8508-a04d-4107-9689-9f3eba082ffe admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:58.206225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df8508-a04d-4107-9689-9f3eba082ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:58.206506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df8508-a04d-4107-9689-9f3eba082ffe admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:58.207176 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df8508-a04d-4107-9689-9f3eba082ffe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:19:58.207483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df8508-a04d-4107-9689-9f3eba082ffe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:58.209183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df8508-a04d-4107-9689-9f3eba082ffe admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:58.222075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 998/1962] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:19:58.227066 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf902d8-2e2e-47e5-beea-dc9d60277803 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:58.237904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf902d8-2e2e-47e5-beea-dc9d60277803 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:58.242568 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a539ba-8194-48d7-a071-6c37b31aa842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:58.243171 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a539ba-8194-48d7-a071-6c37b31aa842 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:58.243454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a539ba-8194-48d7-a071-6c37b31aa842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:58.243728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a539ba-8194-48d7-a071-6c37b31aa842 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:58.244353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a539ba-8194-48d7-a071-6c37b31aa842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:58.246432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a539ba-8194-48d7-a071-6c37b31aa842 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:58.333490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 965/1963] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:19:58 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:58.334856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df8508-a04d-4107-9689-9f3eba082ffe admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:58.347414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df8508-a04d-4107-9689-9f3eba082ffe admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:58.409416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097e68d-0944-4f61-b433-129efc480894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:58.410027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097e68d-0944-4f61-b433-129efc480894 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:58.410351 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097e68d-0944-4f61-b433-129efc480894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:58.410631 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097e68d-0944-4f61-b433-129efc480894 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:58.411258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097e68d-0944-4f61-b433-129efc480894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:58.413220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097e68d-0944-4f61-b433-129efc480894 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:58.505735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 999/1964] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:19:58 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:19:58.508694 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a539ba-8194-48d7-a071-6c37b31aa842 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:58.529390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a539ba-8194-48d7-a071-6c37b31aa842 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:58.534776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228cc60-3cc5-41a1-aa4c-21f261a9ae7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:58.536089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228cc60-3cc5-41a1-aa4c-21f261a9ae7c admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:58.536089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228cc60-3cc5-41a1-aa4c-21f261a9ae7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:58.536089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228cc60-3cc5-41a1-aa4c-21f261a9ae7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:58.536807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228cc60-3cc5-41a1-aa4c-21f261a9ae7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:58.538403 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228cc60-3cc5-41a1-aa4c-21f261a9ae7c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:58.553659 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 966/1965] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:58.555985 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097e68d-0944-4f61-b433-129efc480894 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:58.566666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097e68d-0944-4f61-b433-129efc480894 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:58.571268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3bc70-f033-4359-946a-7ab88e447495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:58.571852 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3bc70-f033-4359-946a-7ab88e447495 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:58.572194 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3bc70-f033-4359-946a-7ab88e447495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:58.572466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3bc70-f033-4359-946a-7ab88e447495 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:58.573138 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3bc70-f033-4359-946a-7ab88e447495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:58.574834 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3bc70-f033-4359-946a-7ab88e447495 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:58.587978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1000/1966] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:19:58.592131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228cc60-3cc5-41a1-aa4c-21f261a9ae7c admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:58.602678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228cc60-3cc5-41a1-aa4c-21f261a9ae7c admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:58.607565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2086b-6264-40da-8b7d-f0c635b57db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:58.608222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2086b-6264-40da-8b7d-f0c635b57db4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:58.610426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2086b-6264-40da-8b7d-f0c635b57db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:58.610426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2086b-6264-40da-8b7d-f0c635b57db4 admin admin] PATH_INFO: `/v3/projects/39e260f3108d4b9f85088dfd7fbf460d/users/cc511731cfaf4836b0671146dd4e71bf/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:58.625251 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb2086b-6264-40da-8b7d-f0c635b57db4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e260f3108d4b9f85088dfd7fbf460d, user_id=cc511731cfaf4836b0671146dd4e71bf, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:58.627939 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb2086b-6264-40da-8b7d-f0c635b57db4 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:58.714815 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 967/1967] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:19:58 2022] PUT /identity/v3/projects/39e260f3108d4b9f85088dfd7fbf460d/users/cc511731cfaf4836b0671146dd4e71bf/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:19:58.726118 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5322c-ab48-4fea-8a7a-1b6635a7a5a3 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:58.726430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5322c-ab48-4fea-8a7a-1b6635a7a5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:58.726700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5322c-ab48-4fea-8a7a-1b6635a7a5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:58.753875 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-12c5322c-ab48-4fea-8a7a-1b6635a7a5a3 None None] MFA Rules not processed for user `cc511731cfaf4836b0671146dd4e71bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:19:58.755122 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-12c5322c-ab48-4fea-8a7a-1b6635a7a5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:58.823550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1001/1968] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:19:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:19:58.835944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2086b-6264-40da-8b7d-f0c635b57db4 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:19:58.837574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9bb2086b-6264-40da-8b7d-f0c635b57db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:19:58.904811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2086b-6264-40da-8b7d-f0c635b57db4 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:19:58.904811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e5969-f1cb-4c5c-8c46-e2dd2f468936 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:19:58.904811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e5969-f1cb-4c5c-8c46-e2dd2f468936 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:19:58.904811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e5969-f1cb-4c5c-8c46-e2dd2f468936 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:19:58.905587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e5969-f1cb-4c5c-8c46-e2dd2f468936 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:19:58.906980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e5969-f1cb-4c5c-8c46-e2dd2f468936 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:19:58.908506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e5969-f1cb-4c5c-8c46-e2dd2f468936 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:19:58.917900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 968/1969] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:19:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:20:04.774735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5322c-ab48-4fea-8a7a-1b6635a7a5a3 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:04.776076 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-12c5322c-ab48-4fea-8a7a-1b6635a7a5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:04.790089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5322c-ab48-4fea-8a7a-1b6635a7a5a3 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:04.794088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04574c8b-99db-43d9-a09a-45ec3cbfa4a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:04.800075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04574c8b-99db-43d9-a09a-45ec3cbfa4a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:04.800075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04574c8b-99db-43d9-a09a-45ec3cbfa4a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:04.800075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04574c8b-99db-43d9-a09a-45ec3cbfa4a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:04.800075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-04574c8b-99db-43d9-a09a-45ec3cbfa4a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:04.854441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:04.878576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04574c8b-99db-43d9-a09a-45ec3cbfa4a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:04.880027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04574c8b-99db-43d9-a09a-45ec3cbfa4a9 service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:04.896565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1002/1970] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:04 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:20:08.913656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:15.116260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=10 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:28.472049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e5969-f1cb-4c5c-8c46-e2dd2f468936 service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:28.481249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4f6e5969-f1cb-4c5c-8c46-e2dd2f468936 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:28.540990 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e5969-f1cb-4c5c-8c46-e2dd2f468936 service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:28.546036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a40b83-130f-46cb-9b22-9c8a03928487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:28.546883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a40b83-130f-46cb-9b22-9c8a03928487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:28.547168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a40b83-130f-46cb-9b22-9c8a03928487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:28.547445 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a40b83-130f-46cb-9b22-9c8a03928487 admin admin] PATH_INFO: `/v3/users/b88dd5b396354ff9a4effb30e4ac8572` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:28.548622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a40b83-130f-46cb-9b22-9c8a03928487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b88dd5b396354ff9a4effb30e4ac8572)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:28.550621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a40b83-130f-46cb-9b22-9c8a03928487 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:28.754475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-23a40b83-130f-46cb-9b22-9c8a03928487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b88dd5b396354ff9a4effb30e4ac8572'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:28.755421 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-23a40b83-130f-46cb-9b22-9c8a03928487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b88dd5b396354ff9a4effb30e4ac8572'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:28.823137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-23a40b83-130f-46cb-9b22-9c8a03928487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b88dd5b396354ff9a4effb30e4ac8572'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:28.840982 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-23a40b83-130f-46cb-9b22-9c8a03928487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b88dd5b396354ff9a4effb30e4ac8572'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:28.849829 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-23a40b83-130f-46cb-9b22-9c8a03928487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b88dd5b396354ff9a4effb30e4ac8572'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:28.857985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 969/1971] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:20:28 2022] DELETE /identity/v3/users/b88dd5b396354ff9a4effb30e4ac8572 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:20:28.863026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a1fd1-1ee7-46c8-a967-58445a355435 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:28.863522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a1fd1-1ee7-46c8-a967-58445a355435 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:28.863602 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a1fd1-1ee7-46c8-a967-58445a355435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:28.888879 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-719a1fd1-1ee7-46c8-a967-58445a355435 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:28.888879 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-719a1fd1-1ee7-46c8-a967-58445a355435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:28.961591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1003/1972] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:28.978768 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a40b83-130f-46cb-9b22-9c8a03928487 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:28.980489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-23a40b83-130f-46cb-9b22-9c8a03928487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:29.044826 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a40b83-130f-46cb-9b22-9c8a03928487 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:29.048573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb84fed-23b4-4cd5-91fc-637b8c8c0ac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:29.049950 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb84fed-23b4-4cd5-91fc-637b8c8c0ac1 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:29.049950 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb84fed-23b4-4cd5-91fc-637b8c8c0ac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:29.049950 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb84fed-23b4-4cd5-91fc-637b8c8c0ac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:29.059003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb84fed-23b4-4cd5-91fc-637b8c8c0ac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:29.060423 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb84fed-23b4-4cd5-91fc-637b8c8c0ac1 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:29.070477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 970/1973] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:20:29.608369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a1fd1-1ee7-46c8-a967-58445a355435 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:29.610152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-719a1fd1-1ee7-46c8-a967-58445a355435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:29.623384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a1fd1-1ee7-46c8-a967-58445a355435 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:29.629672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397550db-0f0a-4268-9817-62da47d01b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:29.630406 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397550db-0f0a-4268-9817-62da47d01b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:29.630859 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397550db-0f0a-4268-9817-62da47d01b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:29.631146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397550db-0f0a-4268-9817-62da47d01b41 admin admin] PATH_INFO: `/v3/projects/4fdd6741d3694ceda2655dc72b5cd75e` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:29.632287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397550db-0f0a-4268-9817-62da47d01b41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fdd6741d3694ceda2655dc72b5cd75e)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:29.634242 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397550db-0f0a-4268-9817-62da47d01b41 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:29.771845 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-397550db-0f0a-4268-9817-62da47d01b41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fdd6741d3694ceda2655dc72b5cd75e', 'request_id': 'req-397550db-0f0a-4268-9817-62da47d01b41'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:29.777497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-397550db-0f0a-4268-9817-62da47d01b41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fdd6741d3694ceda2655dc72b5cd75e', 'request_id': 'req-397550db-0f0a-4268-9817-62da47d01b41'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:29.811629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-397550db-0f0a-4268-9817-62da47d01b41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fdd6741d3694ceda2655dc72b5cd75e', 'request_id': 'req-397550db-0f0a-4268-9817-62da47d01b41'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:29.916893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb84fed-23b4-4cd5-91fc-637b8c8c0ac1 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:29.918323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1bb84fed-23b4-4cd5-91fc-637b8c8c0ac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:29.931753 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb84fed-23b4-4cd5-91fc-637b8c8c0ac1 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:29.958197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:29.968924 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-397550db-0f0a-4268-9817-62da47d01b41 admin admin] The token cache is being invalidate because project 4fdd6741d3694ceda2655dc72b5cd75e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:20:29.970589 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-397550db-0f0a-4268-9817-62da47d01b41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:29.970589 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-397550db-0f0a-4268-9817-62da47d01b41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:29.972436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1004/1974] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:20:29 2022] DELETE /identity/v3/projects/4fdd6741d3694ceda2655dc72b5cd75e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:29.976349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397550db-0f0a-4268-9817-62da47d01b41 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:29.978220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-397550db-0f0a-4268-9817-62da47d01b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:30.006116 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18130a44-eb90-4c03-ae9e-62024bbfc19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:30.007186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18130a44-eb90-4c03-ae9e-62024bbfc19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:30.007483 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18130a44-eb90-4c03-ae9e-62024bbfc19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:30.007781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18130a44-eb90-4c03-ae9e-62024bbfc19a admin admin] PATH_INFO: `/v3/users/76470494a470481ba07bcb261a3b68e4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:30.009330 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18130a44-eb90-4c03-ae9e-62024bbfc19a admin admin] RBAC: Authorizing `identity:delete_user(user_id=76470494a470481ba07bcb261a3b68e4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:30.011282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18130a44-eb90-4c03-ae9e-62024bbfc19a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:30.049023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397550db-0f0a-4268-9817-62da47d01b41 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:30.054666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e17c2c-fc26-4da6-b6e0-749795608aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:30.055298 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e17c2c-fc26-4da6-b6e0-749795608aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:30.055582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e17c2c-fc26-4da6-b6e0-749795608aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:30.055933 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e17c2c-fc26-4da6-b6e0-749795608aae admin admin] PATH_INFO: `/v3/users/106dddad8d594c3290f436cde1b4e6b8` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:30.057096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e17c2c-fc26-4da6-b6e0-749795608aae admin admin] RBAC: Authorizing `identity:delete_user(user_id=106dddad8d594c3290f436cde1b4e6b8)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:30.059024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e17c2c-fc26-4da6-b6e0-749795608aae admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:30.172710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:30.208370 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-18130a44-eb90-4c03-ae9e-62024bbfc19a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76470494a470481ba07bcb261a3b68e4'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:30.209049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-18130a44-eb90-4c03-ae9e-62024bbfc19a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76470494a470481ba07bcb261a3b68e4'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:30.296080 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-80e17c2c-fc26-4da6-b6e0-749795608aae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '106dddad8d594c3290f436cde1b4e6b8'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:30.306034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-18130a44-eb90-4c03-ae9e-62024bbfc19a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76470494a470481ba07bcb261a3b68e4'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:30.318023 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-18130a44-eb90-4c03-ae9e-62024bbfc19a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76470494a470481ba07bcb261a3b68e4'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:30.324890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-18130a44-eb90-4c03-ae9e-62024bbfc19a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76470494a470481ba07bcb261a3b68e4'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:30.331287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 971/1975] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:20:29 2022] DELETE /identity/v3/users/76470494a470481ba07bcb261a3b68e4 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:20:30.335657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fb6a4-d73a-49b1-8bd4-dce488345d22 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:30.335941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fb6a4-d73a-49b1-8bd4-dce488345d22 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:30.336216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fb6a4-d73a-49b1-8bd4-dce488345d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:30.355798 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-1a1fb6a4-d73a-49b1-8bd4-dce488345d22 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:30.357027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1a1fb6a4-d73a-49b1-8bd4-dce488345d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:30.383410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-80e17c2c-fc26-4da6-b6e0-749795608aae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '106dddad8d594c3290f436cde1b4e6b8'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:30.384067 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-80e17c2c-fc26-4da6-b6e0-749795608aae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '106dddad8d594c3290f436cde1b4e6b8'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:30.392726 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-80e17c2c-fc26-4da6-b6e0-749795608aae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '106dddad8d594c3290f436cde1b4e6b8'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:30.407486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-80e17c2c-fc26-4da6-b6e0-749795608aae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '106dddad8d594c3290f436cde1b4e6b8'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:30.419873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1005/1976] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:20:29 2022] DELETE /identity/v3/users/106dddad8d594c3290f436cde1b4e6b8 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:20:30.424213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 972/1977] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:20:30.433809 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e17c2c-fc26-4da6-b6e0-749795608aae admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:30.435569 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-80e17c2c-fc26-4da6-b6e0-749795608aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:30.495284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e17c2c-fc26-4da6-b6e0-749795608aae admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:30.499482 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e6219-06a8-4575-88a2-663682edf8b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:30.500090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e6219-06a8-4575-88a2-663682edf8b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:30.500392 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e6219-06a8-4575-88a2-663682edf8b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:30.500750 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e6219-06a8-4575-88a2-663682edf8b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:30.563660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e6219-06a8-4575-88a2-663682edf8b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:30.565252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e6219-06a8-4575-88a2-663682edf8b4 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:30.575694 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1006/1978] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:20:31.159293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fb6a4-d73a-49b1-8bd4-dce488345d22 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:31.160778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1a1fb6a4-d73a-49b1-8bd4-dce488345d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:31.187431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fb6a4-d73a-49b1-8bd4-dce488345d22 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:31.192528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502e555-c236-4b5c-91f3-d749962b450e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:31.193209 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502e555-c236-4b5c-91f3-d749962b450e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:31.193551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502e555-c236-4b5c-91f3-d749962b450e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:31.193746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502e555-c236-4b5c-91f3-d749962b450e admin admin] PATH_INFO: `/v3/projects/1cace3c427004de4a29eae1d9dfc2fc1` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:31.194892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502e555-c236-4b5c-91f3-d749962b450e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cace3c427004de4a29eae1d9dfc2fc1)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:31.197591 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502e555-c236-4b5c-91f3-d749962b450e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:31.259288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e6219-06a8-4575-88a2-663682edf8b4 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:31.261026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-c34e6219-06a8-4575-88a2-663682edf8b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:31.277551 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e6219-06a8-4575-88a2-663682edf8b4 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:31.284036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b9823-8e86-4326-9126-e675712a6855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:31.284736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b9823-8e86-4326-9126-e675712a6855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:31.285023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b9823-8e86-4326-9126-e675712a6855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:31.285490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b9823-8e86-4326-9126-e675712a6855 admin admin] PATH_INFO: `/v3/projects/8a52eb0debe5478b976b1d3193a872e2` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:31.286728 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b9823-8e86-4326-9126-e675712a6855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a52eb0debe5478b976b1d3193a872e2)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:31.289068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b9823-8e86-4326-9126-e675712a6855 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:31.317724 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:31.328365 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e502e555-c236-4b5c-91f3-d749962b450e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cace3c427004de4a29eae1d9dfc2fc1', 'request_id': 'req-e502e555-c236-4b5c-91f3-d749962b450e'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:31.336001 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e502e555-c236-4b5c-91f3-d749962b450e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cace3c427004de4a29eae1d9dfc2fc1', 'request_id': 'req-e502e555-c236-4b5c-91f3-d749962b450e'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:31.340862 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e502e555-c236-4b5c-91f3-d749962b450e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cace3c427004de4a29eae1d9dfc2fc1', 'request_id': 'req-e502e555-c236-4b5c-91f3-d749962b450e'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:31.412489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e502e555-c236-4b5c-91f3-d749962b450e admin admin] The token cache is being invalidate because project 1cace3c427004de4a29eae1d9dfc2fc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:20:31.412905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e502e555-c236-4b5c-91f3-d749962b450e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:31.413659 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e502e555-c236-4b5c-91f3-d749962b450e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:31.415494 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 973/1979] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:20:31 2022] DELETE /identity/v3/projects/1cace3c427004de4a29eae1d9dfc2fc1 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:31.421061 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af791bf4-8977-4326-8c73-ad00f2dc6c89 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:31.421901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af791bf4-8977-4326-8c73-ad00f2dc6c89 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:31.422335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af791bf4-8977-4326-8c73-ad00f2dc6c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:31.438265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:31.439665 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-550b9823-8e86-4326-9126-e675712a6855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a52eb0debe5478b976b1d3193a872e2', 'request_id': 'req-550b9823-8e86-4326-9126-e675712a6855'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:31.445689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-550b9823-8e86-4326-9126-e675712a6855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a52eb0debe5478b976b1d3193a872e2', 'request_id': 'req-550b9823-8e86-4326-9126-e675712a6855'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:31.445979 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-af791bf4-8977-4326-8c73-ad00f2dc6c89 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:31.447283 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-af791bf4-8977-4326-8c73-ad00f2dc6c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:31.452783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-550b9823-8e86-4326-9126-e675712a6855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a52eb0debe5478b976b1d3193a872e2', 'request_id': 'req-550b9823-8e86-4326-9126-e675712a6855'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:31.519398 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 974/1980] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:20:31.527077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc6b77-62e2-4fd5-b990-8f58038b0286 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:31.527364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc6b77-62e2-4fd5-b990-8f58038b0286 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:31.527638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc6b77-62e2-4fd5-b990-8f58038b0286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:31.539872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-550b9823-8e86-4326-9126-e675712a6855 admin admin] The token cache is being invalidate because project 8a52eb0debe5478b976b1d3193a872e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:20:31.540448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-550b9823-8e86-4326-9126-e675712a6855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:31.541780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-550b9823-8e86-4326-9126-e675712a6855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:31.544264 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1007/1981] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:20:31 2022] DELETE /identity/v3/projects/8a52eb0debe5478b976b1d3193a872e2 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:20:31.548567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b9823-8e86-4326-9126-e675712a6855 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:31.551429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-550b9823-8e86-4326-9126-e675712a6855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:31.554305 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-6efc6b77-62e2-4fd5-b990-8f58038b0286 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:31.555589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6efc6b77-62e2-4fd5-b990-8f58038b0286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:31.567594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 975/1982] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:31.567777 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b9823-8e86-4326-9126-e675712a6855 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:31.568609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc6b77-62e2-4fd5-b990-8f58038b0286 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:31.571331 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f589727-2542-48ef-823b-40adafabb8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:31.572181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f589727-2542-48ef-823b-40adafabb8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:31.572724 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f589727-2542-48ef-823b-40adafabb8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:31.573136 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f589727-2542-48ef-823b-40adafabb8d7 admin admin] PATH_INFO: `/v3/users/c02bbe18bc7e4f0daa17c9f8e5d02108` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:31.574664 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f589727-2542-48ef-823b-40adafabb8d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c02bbe18bc7e4f0daa17c9f8e5d02108)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:31.577468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f589727-2542-48ef-823b-40adafabb8d7 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:31.580059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc6b77-62e2-4fd5-b990-8f58038b0286 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:31.586126 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbbf89-a285-457d-b874-ccde1e9f9568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:31.586676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbbf89-a285-457d-b874-ccde1e9f9568 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:31.586962 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbbf89-a285-457d-b874-ccde1e9f9568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:31.587320 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbbf89-a285-457d-b874-ccde1e9f9568 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:31.587981 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbbf89-a285-457d-b874-ccde1e9f9568 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:20:31.588295 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbbf89-a285-457d-b874-ccde1e9f9568 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:31.590601 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbbf89-a285-457d-b874-ccde1e9f9568 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:31.604695 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 976/1983] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:20:31.609083 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbbf89-a285-457d-b874-ccde1e9f9568 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:31.622000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbbf89-a285-457d-b874-ccde1e9f9568 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:31.630401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f78943-5285-430a-a949-1ffb59c9de6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:31.630401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f78943-5285-430a-a949-1ffb59c9de6a admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:31.630401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f78943-5285-430a-a949-1ffb59c9de6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:31.630401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f78943-5285-430a-a949-1ffb59c9de6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:31.630775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f78943-5285-430a-a949-1ffb59c9de6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:31.632194 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f78943-5285-430a-a949-1ffb59c9de6a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:31.761739 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 977/1984] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:20:31 2022] POST /identity/v3/projects => generated 393 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:20:31.766396 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f78943-5285-430a-a949-1ffb59c9de6a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:31.862809 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f78943-5285-430a-a949-1ffb59c9de6a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:31.863737 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1f589727-2542-48ef-823b-40adafabb8d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c02bbe18bc7e4f0daa17c9f8e5d02108'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:31.868077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b579f-03c9-446a-893b-e7667e9e1689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:31.868747 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b579f-03c9-446a-893b-e7667e9e1689 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:31.869713 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b579f-03c9-446a-893b-e7667e9e1689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:31.869997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b579f-03c9-446a-893b-e7667e9e1689 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:31.870635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b579f-03c9-446a-893b-e7667e9e1689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:31.872663 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b579f-03c9-446a-893b-e7667e9e1689 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:31.978072 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1f589727-2542-48ef-823b-40adafabb8d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c02bbe18bc7e4f0daa17c9f8e5d02108'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:31.978993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1f589727-2542-48ef-823b-40adafabb8d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c02bbe18bc7e4f0daa17c9f8e5d02108'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:31.984239 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 978/1985] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:20:31 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:20:31.988632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b579f-03c9-446a-893b-e7667e9e1689 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:31.990558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1f589727-2542-48ef-823b-40adafabb8d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c02bbe18bc7e4f0daa17c9f8e5d02108'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:31.992597 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-554b579f-03c9-446a-893b-e7667e9e1689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:32.012518 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1f589727-2542-48ef-823b-40adafabb8d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c02bbe18bc7e4f0daa17c9f8e5d02108'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:32.016176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b579f-03c9-446a-893b-e7667e9e1689 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:32.019817 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379109c-95ea-40a7-a5a7-6a4a985a1be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:32.020440 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1008/1986] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:20:31 2022] DELETE /identity/v3/users/c02bbe18bc7e4f0daa17c9f8e5d02108 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:20:32.020688 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379109c-95ea-40a7-a5a7-6a4a985a1be6 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:32.021067 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379109c-95ea-40a7-a5a7-6a4a985a1be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:32.021463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379109c-95ea-40a7-a5a7-6a4a985a1be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:32.022209 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379109c-95ea-40a7-a5a7-6a4a985a1be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:32.024135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379109c-95ea-40a7-a5a7-6a4a985a1be6 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:32.037564 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 979/1987] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:32.041441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f589727-2542-48ef-823b-40adafabb8d7 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:32.122872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f589727-2542-48ef-823b-40adafabb8d7 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:32.129289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf4088-ecd0-400c-a414-0f5a0a35453f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:32.131313 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf4088-ecd0-400c-a414-0f5a0a35453f admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:32.131313 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf4088-ecd0-400c-a414-0f5a0a35453f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:32.131313 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf4088-ecd0-400c-a414-0f5a0a35453f admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:32.131656 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf4088-ecd0-400c-a414-0f5a0a35453f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:32.133084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf4088-ecd0-400c-a414-0f5a0a35453f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:32.144938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1009/1988] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:32.149186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379109c-95ea-40a7-a5a7-6a4a985a1be6 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:32.154703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:32.160368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379109c-95ea-40a7-a5a7-6a4a985a1be6 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:32.165875 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa0b91-d10a-4572-bc1b-f4dbd4dd0480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:32.166680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa0b91-d10a-4572-bc1b-f4dbd4dd0480 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:32.167116 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa0b91-d10a-4572-bc1b-f4dbd4dd0480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:32.167467 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa0b91-d10a-4572-bc1b-f4dbd4dd0480 admin admin] PATH_INFO: `/v3/projects/4b32888ec9554d878c5393719419702a/users/45f7041cf80e491180eca8e6be51f656/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:32.188069 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa0b91-d10a-4572-bc1b-f4dbd4dd0480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b32888ec9554d878c5393719419702a, user_id=45f7041cf80e491180eca8e6be51f656, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:32.191377 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa0b91-d10a-4572-bc1b-f4dbd4dd0480 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:32.290866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 980/1989] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:20:32 2022] PUT /identity/v3/projects/4b32888ec9554d878c5393719419702a/users/45f7041cf80e491180eca8e6be51f656/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:32.303516 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692d567-eb42-489b-bcfe-70c609f0c887 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:32.304091 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692d567-eb42-489b-bcfe-70c609f0c887 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:32.304170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692d567-eb42-489b-bcfe-70c609f0c887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:32.355021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-3692d567-eb42-489b-bcfe-70c609f0c887 None None] MFA Rules not processed for user `45f7041cf80e491180eca8e6be51f656`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:32.356456 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-3692d567-eb42-489b-bcfe-70c609f0c887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:32.429115 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:32.513078 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1010/1990] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:32.521812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa0b91-d10a-4572-bc1b-f4dbd4dd0480 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:32.598369 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa0b91-d10a-4572-bc1b-f4dbd4dd0480 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:32.604292 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac11618-7798-4c7f-b718-48e1bd3252e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:32.605753 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac11618-7798-4c7f-b718-48e1bd3252e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:32.606178 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac11618-7798-4c7f-b718-48e1bd3252e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:32.606569 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac11618-7798-4c7f-b718-48e1bd3252e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:32.607415 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac11618-7798-4c7f-b718-48e1bd3252e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:32.609470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac11618-7798-4c7f-b718-48e1bd3252e3 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:32.705995 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 981/1991] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:20:32 2022] POST /identity/v3/projects => generated 393 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:32.709979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692d567-eb42-489b-bcfe-70c609f0c887 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:32.725033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692d567-eb42-489b-bcfe-70c609f0c887 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:32.763344 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac11618-7798-4c7f-b718-48e1bd3252e3 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:32.766958 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4ac11618-7798-4c7f-b718-48e1bd3252e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:32.820311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b318fbe-1b28-478c-8b6c-f36ace95f052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:32.821258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b318fbe-1b28-478c-8b6c-f36ace95f052 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:32.821710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b318fbe-1b28-478c-8b6c-f36ace95f052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:32.822157 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b318fbe-1b28-478c-8b6c-f36ace95f052 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:32.823978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b318fbe-1b28-478c-8b6c-f36ace95f052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:32.826233 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b318fbe-1b28-478c-8b6c-f36ace95f052 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:32.856926 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac11618-7798-4c7f-b718-48e1bd3252e3 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:32.862547 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8fc95-8fb1-4dc7-af7e-d271b2b5aa93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:32.863442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8fc95-8fb1-4dc7-af7e-d271b2b5aa93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:32.863442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8fc95-8fb1-4dc7-af7e-d271b2b5aa93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:32.863947 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8fc95-8fb1-4dc7-af7e-d271b2b5aa93 admin admin] PATH_INFO: `/v3/projects/1316854811d04a94a359e7b103bc168c` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:32.865306 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8fc95-8fb1-4dc7-af7e-d271b2b5aa93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1316854811d04a94a359e7b103bc168c)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:32.867554 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8fc95-8fb1-4dc7-af7e-d271b2b5aa93 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:32.930249 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1011/1992] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:20:32 2022] POST /identity/v3/users => generated 303 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:32.932023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b318fbe-1b28-478c-8b6c-f36ace95f052 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:32.952791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b318fbe-1b28-478c-8b6c-f36ace95f052 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:32.963123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ae394-beca-417e-b950-8a2ddbb8304a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:32.963742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ae394-beca-417e-b950-8a2ddbb8304a admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:32.964054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ae394-beca-417e-b950-8a2ddbb8304a admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:32.964373 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ae394-beca-417e-b950-8a2ddbb8304a admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:32.965174 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ae394-beca-417e-b950-8a2ddbb8304a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:32.967455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ae394-beca-417e-b950-8a2ddbb8304a admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:32.980680 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1012/1993] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:32.990422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ae394-beca-417e-b950-8a2ddbb8304a admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:33.001580 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ae394-beca-417e-b950-8a2ddbb8304a admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:33.006996 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c82c66-edb4-40da-aa09-d4d0ee750686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:33.007743 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c82c66-edb4-40da-aa09-d4d0ee750686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:33.008103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c82c66-edb4-40da-aa09-d4d0ee750686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:33.009152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c82c66-edb4-40da-aa09-d4d0ee750686 admin admin] PATH_INFO: `/v3/projects/1a5efafea5a1422f9672ef3c5ed38299/users/9e71e24b2490474d8be9abaef687e732/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:33.026508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c82c66-edb4-40da-aa09-d4d0ee750686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5efafea5a1422f9672ef3c5ed38299, user_id=9e71e24b2490474d8be9abaef687e732, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:33.030213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c82c66-edb4-40da-aa09-d4d0ee750686 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:33.033723 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b6f8fc95-8fb1-4dc7-af7e-d271b2b5aa93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1316854811d04a94a359e7b103bc168c', 'request_id': 'req-b6f8fc95-8fb1-4dc7-af7e-d271b2b5aa93'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:33.040876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b6f8fc95-8fb1-4dc7-af7e-d271b2b5aa93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1316854811d04a94a359e7b103bc168c', 'request_id': 'req-b6f8fc95-8fb1-4dc7-af7e-d271b2b5aa93'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:33.049878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b6f8fc95-8fb1-4dc7-af7e-d271b2b5aa93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1316854811d04a94a359e7b103bc168c', 'request_id': 'req-b6f8fc95-8fb1-4dc7-af7e-d271b2b5aa93'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:33.132535 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b6f8fc95-8fb1-4dc7-af7e-d271b2b5aa93 admin admin] The token cache is being invalidate because project 1316854811d04a94a359e7b103bc168c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:20:33.132926 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b6f8fc95-8fb1-4dc7-af7e-d271b2b5aa93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:33.133555 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1013/1994] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:20:32 2022] PUT /identity/v3/projects/1a5efafea5a1422f9672ef3c5ed38299/users/9e71e24b2490474d8be9abaef687e732/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:33.136557 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b6f8fc95-8fb1-4dc7-af7e-d271b2b5aa93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:33.137642 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c82c66-edb4-40da-aa09-d4d0ee750686 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:33.138715 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 982/1995] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:20:32 2022] DELETE /identity/v3/projects/1316854811d04a94a359e7b103bc168c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:33.143744 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-39c82c66-edb4-40da-aa09-d4d0ee750686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:33.144326 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1946b-9a13-44f5-8d19-4e128e11d089 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:33.144713 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1946b-9a13-44f5-8d19-4e128e11d089 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:33.145063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1946b-9a13-44f5-8d19-4e128e11d089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:33.167768 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-26c1946b-9a13-44f5-8d19-4e128e11d089 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:33.169028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-26c1946b-9a13-44f5-8d19-4e128e11d089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:33.222936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c82c66-edb4-40da-aa09-d4d0ee750686 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:33.228460 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07a728-31ce-4b12-813c-99e6bdf0d938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:33.229129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07a728-31ce-4b12-813c-99e6bdf0d938 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:33.229412 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07a728-31ce-4b12-813c-99e6bdf0d938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:33.229694 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07a728-31ce-4b12-813c-99e6bdf0d938 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:33.230394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07a728-31ce-4b12-813c-99e6bdf0d938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:33.232116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07a728-31ce-4b12-813c-99e6bdf0d938 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:33.238293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:33.261987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 983/1996] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:33.269403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1014/1997] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:33.271537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484001c-d9ba-48c8-95da-6251ee0d408b None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:33.272017 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484001c-d9ba-48c8-95da-6251ee0d408b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:33.272838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484001c-d9ba-48c8-95da-6251ee0d408b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:33.272984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07a728-31ce-4b12-813c-99e6bdf0d938 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:33.285101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07a728-31ce-4b12-813c-99e6bdf0d938 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:33.290410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc79f6-48e1-4d74-a60d-9966b20ce98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:33.291165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc79f6-48e1-4d74-a60d-9966b20ce98f admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:33.291569 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc79f6-48e1-4d74-a60d-9966b20ce98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:33.292162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc79f6-48e1-4d74-a60d-9966b20ce98f admin admin] PATH_INFO: `/v3/projects/1a5efafea5a1422f9672ef3c5ed38299/users/9e71e24b2490474d8be9abaef687e732/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:33.294265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc79f6-48e1-4d74-a60d-9966b20ce98f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5efafea5a1422f9672ef3c5ed38299, user_id=9e71e24b2490474d8be9abaef687e732, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:33.294431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-1484001c-d9ba-48c8-95da-6251ee0d408b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:33.296861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1484001c-d9ba-48c8-95da-6251ee0d408b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:33.298371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc79f6-48e1-4d74-a60d-9966b20ce98f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:33.305540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 984/1998] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:33.312670 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1015/1999] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:20:33 2022] PUT /identity/v3/projects/1a5efafea5a1422f9672ef3c5ed38299/users/9e71e24b2490474d8be9abaef687e732/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:33.322810 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484001c-d9ba-48c8-95da-6251ee0d408b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:33.329559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee4ac7-79b4-4aee-9229-c26d03e462db None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:33.330407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee4ac7-79b4-4aee-9229-c26d03e462db None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:33.330741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee4ac7-79b4-4aee-9229-c26d03e462db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:33.334441 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484001c-d9ba-48c8-95da-6251ee0d408b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:33.366783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9fee4ac7-79b4-4aee-9229-c26d03e462db None None] MFA Rules not processed for user `9e71e24b2490474d8be9abaef687e732`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:33.368067 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9fee4ac7-79b4-4aee-9229-c26d03e462db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:33.402244 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124865d4-4357-4cdb-beec-df817baec316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:33.405522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124865d4-4357-4cdb-beec-df817baec316 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:33.406706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124865d4-4357-4cdb-beec-df817baec316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:33.407383 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124865d4-4357-4cdb-beec-df817baec316 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:33.408359 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124865d4-4357-4cdb-beec-df817baec316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:20:33.408849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124865d4-4357-4cdb-beec-df817baec316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:33.414621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124865d4-4357-4cdb-beec-df817baec316 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:33.431673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 985/2000] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:20:33.437278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124865d4-4357-4cdb-beec-df817baec316 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:33.452201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124865d4-4357-4cdb-beec-df817baec316 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:33.457182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ede09-e034-476d-a365-cb632c051d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:33.457968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ede09-e034-476d-a365-cb632c051d2d admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:33.458324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ede09-e034-476d-a365-cb632c051d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:33.458664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ede09-e034-476d-a365-cb632c051d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:33.459357 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ede09-e034-476d-a365-cb632c051d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:33.461446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ede09-e034-476d-a365-cb632c051d2d admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:33.474899 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1016/2001] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:33 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:33.484428 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee4ac7-79b4-4aee-9229-c26d03e462db None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:33.485961 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9fee4ac7-79b4-4aee-9229-c26d03e462db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:33.545532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee4ac7-79b4-4aee-9229-c26d03e462db None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:33.549489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2978e-f41e-41e1-8a0a-913beb22ef9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:33.550103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2978e-f41e-41e1-8a0a-913beb22ef9c service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:33.550386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2978e-f41e-41e1-8a0a-913beb22ef9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:33.550655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2978e-f41e-41e1-8a0a-913beb22ef9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:33.559892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2978e-f41e-41e1-8a0a-913beb22ef9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:33.562186 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2978e-f41e-41e1-8a0a-913beb22ef9c service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:33.572993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1017/2002] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:33 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:20:33.582841 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 986/2003] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:20:33 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:33.584913 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2978e-f41e-41e1-8a0a-913beb22ef9c service cinder] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:33.596007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2978e-f41e-41e1-8a0a-913beb22ef9c service cinder] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:33.664112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d658f5-08ae-471f-99fe-44bfabc2f754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:33.664775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d658f5-08ae-471f-99fe-44bfabc2f754 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:33.665064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d658f5-08ae-471f-99fe-44bfabc2f754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:33.665346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d658f5-08ae-471f-99fe-44bfabc2f754 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:33.666035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d658f5-08ae-471f-99fe-44bfabc2f754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:33.668499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d658f5-08ae-471f-99fe-44bfabc2f754 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:33.694428 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ede09-e034-476d-a365-cb632c051d2d admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:33.759141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ede09-e034-476d-a365-cb632c051d2d admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:33.763310 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499b568-4e2b-423d-ae32-17a9b7753425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:33.764690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499b568-4e2b-423d-ae32-17a9b7753425 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:33.764982 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499b568-4e2b-423d-ae32-17a9b7753425 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:33.765253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499b568-4e2b-423d-ae32-17a9b7753425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:33.767820 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1499b568-4e2b-423d-ae32-17a9b7753425 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:33.782127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1018/2004] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:20:33 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:33.784465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d658f5-08ae-471f-99fe-44bfabc2f754 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:33.804887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d658f5-08ae-471f-99fe-44bfabc2f754 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:33.825721 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817570fa-0e29-4f93-a2bf-158d917242ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:33.826580 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817570fa-0e29-4f93-a2bf-158d917242ee admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:33.826874 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817570fa-0e29-4f93-a2bf-158d917242ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:33.827213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817570fa-0e29-4f93-a2bf-158d917242ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:33.828380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817570fa-0e29-4f93-a2bf-158d917242ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:33.829585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817570fa-0e29-4f93-a2bf-158d917242ee admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:33.841307 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1019/2005] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:33.845503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817570fa-0e29-4f93-a2bf-158d917242ee admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:33.855250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499b568-4e2b-423d-ae32-17a9b7753425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:33.858186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1499b568-4e2b-423d-ae32-17a9b7753425 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:33.858487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817570fa-0e29-4f93-a2bf-158d917242ee admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:33.864276 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e66633-b583-4412-8d7b-2b10b6297ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:33.865084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e66633-b583-4412-8d7b-2b10b6297ffc admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:33.866153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e66633-b583-4412-8d7b-2b10b6297ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:33.866543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e66633-b583-4412-8d7b-2b10b6297ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:33.867426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e66633-b583-4412-8d7b-2b10b6297ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:33.869561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e66633-b583-4412-8d7b-2b10b6297ffc admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:33.871610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 987/2006] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:33 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:20:33.889477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1020/2007] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:33.893468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499b568-4e2b-423d-ae32-17a9b7753425 service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:33.908732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499b568-4e2b-423d-ae32-17a9b7753425 service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:33.916542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89c859-df03-4478-b477-b9f8311fe100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:33.918449 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89c859-df03-4478-b477-b9f8311fe100 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:33.919393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89c859-df03-4478-b477-b9f8311fe100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:33.919393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89c859-df03-4478-b477-b9f8311fe100 admin admin] PATH_INFO: `/v3/projects/288735a6a68545fba85cb5a26a15c038/users/20fd926963cc440cb50df97c158b1a67/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:33.940854 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89c859-df03-4478-b477-b9f8311fe100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=288735a6a68545fba85cb5a26a15c038, user_id=20fd926963cc440cb50df97c158b1a67, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:33.944121 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89c859-df03-4478-b477-b9f8311fe100 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:34.026004 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 988/2008] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:20:33 2022] PUT /identity/v3/projects/288735a6a68545fba85cb5a26a15c038/users/20fd926963cc440cb50df97c158b1a67/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:20:34.038582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc770fa-668f-4ec9-a56d-dcdbe794ddb4 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:34.039143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc770fa-668f-4ec9-a56d-dcdbe794ddb4 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:34.039420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc770fa-668f-4ec9-a56d-dcdbe794ddb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:34.070978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-5dc770fa-668f-4ec9-a56d-dcdbe794ddb4 None None] MFA Rules not processed for user `20fd926963cc440cb50df97c158b1a67`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:34.072227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-5dc770fa-668f-4ec9-a56d-dcdbe794ddb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:34.147151 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1021/2009] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:34.252080 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:34.650168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89c859-df03-4478-b477-b9f8311fe100 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:34.711179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89c859-df03-4478-b477-b9f8311fe100 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:34.721791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb3e19-6345-44e7-8ef3-ed10ab17251e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:34.721791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb3e19-6345-44e7-8ef3-ed10ab17251e service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:34.721791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb3e19-6345-44e7-8ef3-ed10ab17251e service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:34.721791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb3e19-6345-44e7-8ef3-ed10ab17251e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:34.726911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb3e19-6345-44e7-8ef3-ed10ab17251e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:34.728448 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb3e19-6345-44e7-8ef3-ed10ab17251e service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:34.739364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 989/2010] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:20:34.762285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc770fa-668f-4ec9-a56d-dcdbe794ddb4 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:34.826115 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:34.858134 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc770fa-668f-4ec9-a56d-dcdbe794ddb4 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:34.879095 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538317b8-4ed7-4b35-bb81-8506b909b8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:34.886013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538317b8-4ed7-4b35-bb81-8506b909b8d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:34.886013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538317b8-4ed7-4b35-bb81-8506b909b8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:34.886013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538317b8-4ed7-4b35-bb81-8506b909b8d0 admin admin] PATH_INFO: `/v3/users/20fd926963cc440cb50df97c158b1a67` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:34.886013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538317b8-4ed7-4b35-bb81-8506b909b8d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20fd926963cc440cb50df97c158b1a67)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:34.886013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538317b8-4ed7-4b35-bb81-8506b909b8d0 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:35.134770 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-538317b8-4ed7-4b35-bb81-8506b909b8d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20fd926963cc440cb50df97c158b1a67'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:35.154867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:35.205418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-538317b8-4ed7-4b35-bb81-8506b909b8d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20fd926963cc440cb50df97c158b1a67'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:35.205418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-538317b8-4ed7-4b35-bb81-8506b909b8d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20fd926963cc440cb50df97c158b1a67'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:35.211268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-538317b8-4ed7-4b35-bb81-8506b909b8d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20fd926963cc440cb50df97c158b1a67'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:35.223829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-538317b8-4ed7-4b35-bb81-8506b909b8d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20fd926963cc440cb50df97c158b1a67'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:35.230061 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1022/2011] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:20:34 2022] DELETE /identity/v3/users/20fd926963cc440cb50df97c158b1a67 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:35.234910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe6374-b941-467c-9349-35c74d4ad64e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:35.235205 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe6374-b941-467c-9349-35c74d4ad64e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:35.235464 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe6374-b941-467c-9349-35c74d4ad64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:35.255196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-1fbe6374-b941-467c-9349-35c74d4ad64e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:35.256122 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1fbe6374-b941-467c-9349-35c74d4ad64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:35.317354 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 990/2012] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:35.328068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538317b8-4ed7-4b35-bb81-8506b909b8d0 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:35.330156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-538317b8-4ed7-4b35-bb81-8506b909b8d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:35.403507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538317b8-4ed7-4b35-bb81-8506b909b8d0 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:35.408347 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0edc2-1b17-4c21-b518-63960709b2dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:35.409230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0edc2-1b17-4c21-b518-63960709b2dd service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:35.414336 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0edc2-1b17-4c21-b518-63960709b2dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:35.414336 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0edc2-1b17-4c21-b518-63960709b2dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:35.420579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0edc2-1b17-4c21-b518-63960709b2dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:35.422123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0edc2-1b17-4c21-b518-63960709b2dd service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:35.432322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1023/2013] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:20:35.878645 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe6374-b941-467c-9349-35c74d4ad64e None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:35.882880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1fbe6374-b941-467c-9349-35c74d4ad64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:35.883251 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:35.893414 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe6374-b941-467c-9349-35c74d4ad64e None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:35.899760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6aaacd-1d55-47b3-bf2c-8ec3fdc30e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:35.900882 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aaacd-1d55-47b3-bf2c-8ec3fdc30e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:35.901381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aaacd-1d55-47b3-bf2c-8ec3fdc30e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:35.901809 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aaacd-1d55-47b3-bf2c-8ec3fdc30e7a admin admin] PATH_INFO: `/v3/projects/288735a6a68545fba85cb5a26a15c038` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:35.905838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6aaacd-1d55-47b3-bf2c-8ec3fdc30e7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=288735a6a68545fba85cb5a26a15c038)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:35.905838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6aaacd-1d55-47b3-bf2c-8ec3fdc30e7a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:36.008207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:36.024406 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2e6aaacd-1d55-47b3-bf2c-8ec3fdc30e7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '288735a6a68545fba85cb5a26a15c038', 'request_id': 'req-2e6aaacd-1d55-47b3-bf2c-8ec3fdc30e7a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:36.030554 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2e6aaacd-1d55-47b3-bf2c-8ec3fdc30e7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '288735a6a68545fba85cb5a26a15c038', 'request_id': 'req-2e6aaacd-1d55-47b3-bf2c-8ec3fdc30e7a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:36.037261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2e6aaacd-1d55-47b3-bf2c-8ec3fdc30e7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '288735a6a68545fba85cb5a26a15c038', 'request_id': 'req-2e6aaacd-1d55-47b3-bf2c-8ec3fdc30e7a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:36.110041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2e6aaacd-1d55-47b3-bf2c-8ec3fdc30e7a admin admin] The token cache is being invalidate because project 288735a6a68545fba85cb5a26a15c038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:20:36.110330 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2e6aaacd-1d55-47b3-bf2c-8ec3fdc30e7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:36.110948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2e6aaacd-1d55-47b3-bf2c-8ec3fdc30e7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:36.112481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 991/2014] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:20:35 2022] DELETE /identity/v3/projects/288735a6a68545fba85cb5a26a15c038 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:20:36.117389 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a116cf-5efd-4e77-b012-a000914f805f None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:36.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a116cf-5efd-4e77-b012-a000914f805f None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:36.119437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a116cf-5efd-4e77-b012-a000914f805f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:36.137193 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-04a116cf-5efd-4e77-b012-a000914f805f None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:36.139137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-04a116cf-5efd-4e77-b012-a000914f805f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:36.196369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1024/2015] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:36.204890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2be207-3ca3-4cfb-b6a9-10b79e472fee None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:36.205176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2be207-3ca3-4cfb-b6a9-10b79e472fee None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:36.205451 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2be207-3ca3-4cfb-b6a9-10b79e472fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:36.225282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-8e2be207-3ca3-4cfb-b6a9-10b79e472fee None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:36.226494 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8e2be207-3ca3-4cfb-b6a9-10b79e472fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:36.234476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 992/2016] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:36.238782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a116cf-5efd-4e77-b012-a000914f805f None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:36.248677 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a116cf-5efd-4e77-b012-a000914f805f None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:36.253382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f88d88-94c3-4953-a877-4517053ad161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:36.253987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f88d88-94c3-4953-a877-4517053ad161 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:36.254265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f88d88-94c3-4953-a877-4517053ad161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:36.254535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f88d88-94c3-4953-a877-4517053ad161 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:36.255212 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f88d88-94c3-4953-a877-4517053ad161 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:20:36.255513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f88d88-94c3-4953-a877-4517053ad161 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:36.257231 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f88d88-94c3-4953-a877-4517053ad161 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:36.269431 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1025/2017] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:20:36.273930 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2be207-3ca3-4cfb-b6a9-10b79e472fee None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:36.284900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2be207-3ca3-4cfb-b6a9-10b79e472fee None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:36.289853 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f0bba-0d7b-4ee0-b836-fe1c7fa3edfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:36.290458 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f0bba-0d7b-4ee0-b836-fe1c7fa3edfc admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:36.290764 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f0bba-0d7b-4ee0-b836-fe1c7fa3edfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:36.291078 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f0bba-0d7b-4ee0-b836-fe1c7fa3edfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:36.291757 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f0bba-0d7b-4ee0-b836-fe1c7fa3edfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:36.293812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f0bba-0d7b-4ee0-b836-fe1c7fa3edfc admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:36.392025 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 993/2018] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:20:36 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:36.396279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f88d88-94c3-4953-a877-4517053ad161 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:36.407096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f88d88-94c3-4953-a877-4517053ad161 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:36.467611 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4105d8b-74d3-4ab0-b327-37d672e7ff6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:36.468239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4105d8b-74d3-4ab0-b327-37d672e7ff6b admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:36.468519 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4105d8b-74d3-4ab0-b327-37d672e7ff6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:36.468823 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4105d8b-74d3-4ab0-b327-37d672e7ff6b admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:36.469454 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4105d8b-74d3-4ab0-b327-37d672e7ff6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:36.471388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4105d8b-74d3-4ab0-b327-37d672e7ff6b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:36.571397 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1026/2019] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:20:36 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:36.580410 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f0bba-0d7b-4ee0-b836-fe1c7fa3edfc admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:36.584563 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f0bba-0d7b-4ee0-b836-fe1c7fa3edfc admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:36.589879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125fc00-6afd-4a92-9d76-51253af45550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:36.590494 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125fc00-6afd-4a92-9d76-51253af45550 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:36.590774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125fc00-6afd-4a92-9d76-51253af45550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:36.591044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125fc00-6afd-4a92-9d76-51253af45550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:36.591843 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d125fc00-6afd-4a92-9d76-51253af45550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:36.593486 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d125fc00-6afd-4a92-9d76-51253af45550 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:36.605751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 994/2020] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:20:36.608536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4105d8b-74d3-4ab0-b327-37d672e7ff6b admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:36.618732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4105d8b-74d3-4ab0-b327-37d672e7ff6b admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:36.624175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670faee-313b-40c6-9bbf-84d28d9567ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:36.624872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670faee-313b-40c6-9bbf-84d28d9567ab admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:36.625158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670faee-313b-40c6-9bbf-84d28d9567ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:36.625431 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670faee-313b-40c6-9bbf-84d28d9567ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:36.626103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670faee-313b-40c6-9bbf-84d28d9567ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:36.627701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670faee-313b-40c6-9bbf-84d28d9567ab admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:36.638130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1027/2021] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:36.642851 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125fc00-6afd-4a92-9d76-51253af45550 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:36.652844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125fc00-6afd-4a92-9d76-51253af45550 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:36.657651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5abf5-ffca-4362-be28-97016aeea658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:36.658304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5abf5-ffca-4362-be28-97016aeea658 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:36.658594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5abf5-ffca-4362-be28-97016aeea658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:36.658871 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5abf5-ffca-4362-be28-97016aeea658 admin admin] PATH_INFO: `/v3/projects/dc7bbebd38c745e0a503ba9a6417b800/users/bd234c7416a142f9b9cf79c2cc8a3672/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:36.673668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5abf5-ffca-4362-be28-97016aeea658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7bbebd38c745e0a503ba9a6417b800, user_id=bd234c7416a142f9b9cf79c2cc8a3672, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:36.676357 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5abf5-ffca-4362-be28-97016aeea658 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:36.783444 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 995/2022] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:20:36 2022] PUT /identity/v3/projects/dc7bbebd38c745e0a503ba9a6417b800/users/bd234c7416a142f9b9cf79c2cc8a3672/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:36.793997 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82505a8e-2cdd-4a80-9abe-5a4f3e1e8d6f None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:36.794284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82505a8e-2cdd-4a80-9abe-5a4f3e1e8d6f None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:36.795218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82505a8e-2cdd-4a80-9abe-5a4f3e1e8d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:36.825170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-82505a8e-2cdd-4a80-9abe-5a4f3e1e8d6f None None] MFA Rules not processed for user `bd234c7416a142f9b9cf79c2cc8a3672`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:36.826981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-82505a8e-2cdd-4a80-9abe-5a4f3e1e8d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:36.839489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:36.893649 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1028/2023] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:36.898060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5abf5-ffca-4362-be28-97016aeea658 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:36.969090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5abf5-ffca-4362-be28-97016aeea658 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:36.976231 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0825cf3-2c52-4083-9088-c007f581b9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:36.976937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0825cf3-2c52-4083-9088-c007f581b9fa admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:36.977225 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0825cf3-2c52-4083-9088-c007f581b9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:36.977501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0825cf3-2c52-4083-9088-c007f581b9fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:36.978146 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0825cf3-2c52-4083-9088-c007f581b9fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:36.980247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0825cf3-2c52-4083-9088-c007f581b9fa admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:37.077985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 996/2024] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:20:36 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:37.081876 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82505a8e-2cdd-4a80-9abe-5a4f3e1e8d6f None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:37.093022 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82505a8e-2cdd-4a80-9abe-5a4f3e1e8d6f None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:37.149675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86279609-8d29-4cc8-9a60-aa0280a8ff68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:37.150275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86279609-8d29-4cc8-9a60-aa0280a8ff68 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:37.150576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86279609-8d29-4cc8-9a60-aa0280a8ff68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:37.150854 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86279609-8d29-4cc8-9a60-aa0280a8ff68 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:37.151475 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86279609-8d29-4cc8-9a60-aa0280a8ff68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:37.153425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86279609-8d29-4cc8-9a60-aa0280a8ff68 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:37.228799 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1029/2025] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:20:37 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:37.233476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0825cf3-2c52-4083-9088-c007f581b9fa admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:37.244012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0825cf3-2c52-4083-9088-c007f581b9fa admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:37.249285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0eac3-e8e1-4859-999f-c750802225ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:37.249922 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0eac3-e8e1-4859-999f-c750802225ef admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:37.250206 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0eac3-e8e1-4859-999f-c750802225ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:37.250523 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0eac3-e8e1-4859-999f-c750802225ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:37.254714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0eac3-e8e1-4859-999f-c750802225ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:37.254714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0eac3-e8e1-4859-999f-c750802225ef admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:37.265169 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 997/2026] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:37.269493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86279609-8d29-4cc8-9a60-aa0280a8ff68 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:37.280455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86279609-8d29-4cc8-9a60-aa0280a8ff68 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:37.285149 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36861c-a448-40e0-9108-e765dd71971c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:37.285791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36861c-a448-40e0-9108-e765dd71971c admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:37.286078 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36861c-a448-40e0-9108-e765dd71971c admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:37.286355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36861c-a448-40e0-9108-e765dd71971c admin admin] PATH_INFO: `/v3/projects/54c5d5a82f8d46749873ea5a7837e128/users/3811d36bc69041ba8076817c8b665314/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:37.302601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36861c-a448-40e0-9108-e765dd71971c admin admin] RBAC: Authorizing `identity:create_grant(project_id=54c5d5a82f8d46749873ea5a7837e128, user_id=3811d36bc69041ba8076817c8b665314, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:37.305322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36861c-a448-40e0-9108-e765dd71971c admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:37.400121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1030/2027] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:20:37 2022] PUT /identity/v3/projects/54c5d5a82f8d46749873ea5a7837e128/users/3811d36bc69041ba8076817c8b665314/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:37.404857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0eac3-e8e1-4859-999f-c750802225ef admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:37.416744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0eac3-e8e1-4859-999f-c750802225ef admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:37.441179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:37.473422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a623898-1038-4f99-914f-d22e6067189d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:37.474016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a623898-1038-4f99-914f-d22e6067189d admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:37.474292 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a623898-1038-4f99-914f-d22e6067189d admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:37.474560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a623898-1038-4f99-914f-d22e6067189d admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:37.475370 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a623898-1038-4f99-914f-d22e6067189d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:37.477353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a623898-1038-4f99-914f-d22e6067189d admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:37.488078 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 998/2028] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:20:37.492322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36861c-a448-40e0-9108-e765dd71971c admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:37.502703 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36861c-a448-40e0-9108-e765dd71971c admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:37.507341 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07a19c-369b-4263-8032-6b3f5d58e67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:37.507977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07a19c-369b-4263-8032-6b3f5d58e67b admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:37.508260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07a19c-369b-4263-8032-6b3f5d58e67b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:37.508538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07a19c-369b-4263-8032-6b3f5d58e67b admin admin] PATH_INFO: `/v3/projects/54c5d5a82f8d46749873ea5a7837e128/users/3811d36bc69041ba8076817c8b665314/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:37.510491 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07a19c-369b-4263-8032-6b3f5d58e67b admin admin] RBAC: Authorizing `identity:create_grant(project_id=54c5d5a82f8d46749873ea5a7837e128, user_id=3811d36bc69041ba8076817c8b665314, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:37.513210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07a19c-369b-4263-8032-6b3f5d58e67b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:37.523338 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1031/2029] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:20:37 2022] PUT /identity/v3/projects/54c5d5a82f8d46749873ea5a7837e128/users/3811d36bc69041ba8076817c8b665314/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:37.535053 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97a85d-ac9a-45b5-a52e-93987d6f34d6 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:37.535336 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97a85d-ac9a-45b5-a52e-93987d6f34d6 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:37.535610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97a85d-ac9a-45b5-a52e-93987d6f34d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:37.563204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-8b97a85d-ac9a-45b5-a52e-93987d6f34d6 None None] MFA Rules not processed for user `3811d36bc69041ba8076817c8b665314`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:37.564388 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8b97a85d-ac9a-45b5-a52e-93987d6f34d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:37.633892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 999/2030] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:37.685853 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07a19c-369b-4263-8032-6b3f5d58e67b admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:37.687761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7d07a19c-369b-4263-8032-6b3f5d58e67b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:37.752623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07a19c-369b-4263-8032-6b3f5d58e67b admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:37.757211 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8ebdc-0818-4868-ae88-17136a6b568e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:37.757823 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8ebdc-0818-4868-ae88-17136a6b568e service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:37.758134 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8ebdc-0818-4868-ae88-17136a6b568e service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:37.758411 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8ebdc-0818-4868-ae88-17136a6b568e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:37.779561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:37.834263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8ebdc-0818-4868-ae88-17136a6b568e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:37.835985 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8ebdc-0818-4868-ae88-17136a6b568e service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:37.846816 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1032/2031] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:20:47.631324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=51 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:47.865726 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=18 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:49.741405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97a85d-ac9a-45b5-a52e-93987d6f34d6 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:49.830722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97a85d-ac9a-45b5-a52e-93987d6f34d6 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:49.833186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2072674-3f0e-4142-8b72-9b81abcdf4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:49.833827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2072674-3f0e-4142-8b72-9b81abcdf4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:49.834116 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2072674-3f0e-4142-8b72-9b81abcdf4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:49.834396 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2072674-3f0e-4142-8b72-9b81abcdf4ee admin admin] PATH_INFO: `/v3/users/bd234c7416a142f9b9cf79c2cc8a3672` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:49.835533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2072674-3f0e-4142-8b72-9b81abcdf4ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd234c7416a142f9b9cf79c2cc8a3672)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:49.837631 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2072674-3f0e-4142-8b72-9b81abcdf4ee admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:50.025501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c2072674-3f0e-4142-8b72-9b81abcdf4ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd234c7416a142f9b9cf79c2cc8a3672'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:50.026217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c2072674-3f0e-4142-8b72-9b81abcdf4ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd234c7416a142f9b9cf79c2cc8a3672'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:50.092779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c2072674-3f0e-4142-8b72-9b81abcdf4ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd234c7416a142f9b9cf79c2cc8a3672'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:50.107190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c2072674-3f0e-4142-8b72-9b81abcdf4ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd234c7416a142f9b9cf79c2cc8a3672'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:50.114288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c2072674-3f0e-4142-8b72-9b81abcdf4ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd234c7416a142f9b9cf79c2cc8a3672'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:50.120278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1000/2032] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:20:49 2022] DELETE /identity/v3/users/bd234c7416a142f9b9cf79c2cc8a3672 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:50.124752 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60241a38-6ade-494f-8979-c2261de52b0d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:50.125159 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60241a38-6ade-494f-8979-c2261de52b0d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:50.125310 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60241a38-6ade-494f-8979-c2261de52b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:50.144603 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-60241a38-6ade-494f-8979-c2261de52b0d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:50.145830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-60241a38-6ade-494f-8979-c2261de52b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:50.203792 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1033/2033] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:50.215123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60241a38-6ade-494f-8979-c2261de52b0d None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:50.221520 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-60241a38-6ade-494f-8979-c2261de52b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:50.222033 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2072674-3f0e-4142-8b72-9b81abcdf4ee admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:50.222746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c2072674-3f0e-4142-8b72-9b81abcdf4ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:50.243560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2072674-3f0e-4142-8b72-9b81abcdf4ee admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:50.252085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6d197-7732-4bff-8326-e9ee9532eb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:50.252085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6d197-7732-4bff-8326-e9ee9532eb8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:50.252085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6d197-7732-4bff-8326-e9ee9532eb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:50.252085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6d197-7732-4bff-8326-e9ee9532eb8f admin admin] PATH_INFO: `/v3/users/45f7041cf80e491180eca8e6be51f656` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:50.252725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6d197-7732-4bff-8326-e9ee9532eb8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=45f7041cf80e491180eca8e6be51f656)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:50.253946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6d197-7732-4bff-8326-e9ee9532eb8f admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:50.301670 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60241a38-6ade-494f-8979-c2261de52b0d None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:50.305298 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d07b6-faae-4d0d-a8ea-dde84b3df8ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:50.305903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d07b6-faae-4d0d-a8ea-dde84b3df8ee service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:50.306184 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d07b6-faae-4d0d-a8ea-dde84b3df8ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:50.306456 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d07b6-faae-4d0d-a8ea-dde84b3df8ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:50.316335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d07b6-faae-4d0d-a8ea-dde84b3df8ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:50.320255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d07b6-faae-4d0d-a8ea-dde84b3df8ee service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:50.327353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1034/2034] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:20:50.487201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-0cc6d197-7732-4bff-8326-e9ee9532eb8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45f7041cf80e491180eca8e6be51f656'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:50.487871 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-0cc6d197-7732-4bff-8326-e9ee9532eb8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45f7041cf80e491180eca8e6be51f656'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:50.591546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-0cc6d197-7732-4bff-8326-e9ee9532eb8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45f7041cf80e491180eca8e6be51f656'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:50.600935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-0cc6d197-7732-4bff-8326-e9ee9532eb8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45f7041cf80e491180eca8e6be51f656'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:50.603261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:50.608699 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-0cc6d197-7732-4bff-8326-e9ee9532eb8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45f7041cf80e491180eca8e6be51f656'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:50.620435 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1001/2035] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:20:50 2022] DELETE /identity/v3/users/45f7041cf80e491180eca8e6be51f656 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:50.621764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4003e5-ee79-4551-b976-12117fb6e77e None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:50.622054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4003e5-ee79-4551-b976-12117fb6e77e None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:50.622512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4003e5-ee79-4551-b976-12117fb6e77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:50.643984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-ea4003e5-ee79-4551-b976-12117fb6e77e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:50.649320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ea4003e5-ee79-4551-b976-12117fb6e77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:50.716981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1035/2036] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:50.726558 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6d197-7732-4bff-8326-e9ee9532eb8f admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:50.727950 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0cc6d197-7732-4bff-8326-e9ee9532eb8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:50.788132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6d197-7732-4bff-8326-e9ee9532eb8f admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:50.791190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4003e5-ee79-4551-b976-12117fb6e77e None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:50.792531 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d3abd-26b9-4d2c-95c0-1db852511731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:50.792754 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ea4003e5-ee79-4551-b976-12117fb6e77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:50.793286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d3abd-26b9-4d2c-95c0-1db852511731 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:50.793629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d3abd-26b9-4d2c-95c0-1db852511731 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:50.793967 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d3abd-26b9-4d2c-95c0-1db852511731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:50.805467 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d3abd-26b9-4d2c-95c0-1db852511731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:50.807667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d3abd-26b9-4d2c-95c0-1db852511731 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:50.809858 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4003e5-ee79-4551-b976-12117fb6e77e None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:50.819413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1002/2037] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:20:50.822138 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:50.822798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:50.823191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:50.823477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3 admin admin] PATH_INFO: `/v3/projects/dc7bbebd38c745e0a503ba9a6417b800` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:50.824710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc7bbebd38c745e0a503ba9a6417b800)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:50.826642 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:50.936656 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:50.943640 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc7bbebd38c745e0a503ba9a6417b800', 'request_id': 'req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:50.949099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc7bbebd38c745e0a503ba9a6417b800', 'request_id': 'req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:50.955811 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc7bbebd38c745e0a503ba9a6417b800', 'request_id': 'req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.027919 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3 admin admin] The token cache is being invalidate because project dc7bbebd38c745e0a503ba9a6417b800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:20:51.028319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.029176 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.030942 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1036/2038] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:20:50 2022] DELETE /identity/v3/projects/dc7bbebd38c745e0a503ba9a6417b800 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:20:51.037555 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d3abd-26b9-4d2c-95c0-1db852511731 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:51.045379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d3abd-26b9-4d2c-95c0-1db852511731 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:51.137320 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e390be-b4cd-47e9-bc8f-087e434c8bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:51.139145 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e390be-b4cd-47e9-bc8f-087e434c8bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:51.139145 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e390be-b4cd-47e9-bc8f-087e434c8bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:51.139145 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e390be-b4cd-47e9-bc8f-087e434c8bcf admin admin] PATH_INFO: `/v3/users/3811d36bc69041ba8076817c8b665314` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:51.139792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e390be-b4cd-47e9-bc8f-087e434c8bcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3811d36bc69041ba8076817c8b665314)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:51.141788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e390be-b4cd-47e9-bc8f-087e434c8bcf admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:51.299804 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:51.301332 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:51.315316 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07ffa0-5326-4d66-8bc6-adf7fff29ee3 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:51.320093 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d70e3-983b-40ea-bde1-0cfa0988b007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:51.320935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d70e3-983b-40ea-bde1-0cfa0988b007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:51.321637 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d70e3-983b-40ea-bde1-0cfa0988b007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:51.321763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d70e3-983b-40ea-bde1-0cfa0988b007 admin admin] PATH_INFO: `/v3/projects/4b32888ec9554d878c5393719419702a` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:51.322897 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d70e3-983b-40ea-bde1-0cfa0988b007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b32888ec9554d878c5393719419702a)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:51.324858 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d70e3-983b-40ea-bde1-0cfa0988b007 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:51.353714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-91e390be-b4cd-47e9-bc8f-087e434c8bcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3811d36bc69041ba8076817c8b665314'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.354359 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-91e390be-b4cd-47e9-bc8f-087e434c8bcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3811d36bc69041ba8076817c8b665314'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.415530 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-91e390be-b4cd-47e9-bc8f-087e434c8bcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3811d36bc69041ba8076817c8b665314'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.425654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-505d70e3-983b-40ea-bde1-0cfa0988b007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b32888ec9554d878c5393719419702a', 'request_id': 'req-505d70e3-983b-40ea-bde1-0cfa0988b007'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.432270 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-505d70e3-983b-40ea-bde1-0cfa0988b007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b32888ec9554d878c5393719419702a', 'request_id': 'req-505d70e3-983b-40ea-bde1-0cfa0988b007'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.438727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-505d70e3-983b-40ea-bde1-0cfa0988b007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b32888ec9554d878c5393719419702a', 'request_id': 'req-505d70e3-983b-40ea-bde1-0cfa0988b007'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.440798 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-91e390be-b4cd-47e9-bc8f-087e434c8bcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3811d36bc69041ba8076817c8b665314'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.448064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-91e390be-b4cd-47e9-bc8f-087e434c8bcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3811d36bc69041ba8076817c8b665314'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.453976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1003/2039] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:20:51 2022] DELETE /identity/v3/users/3811d36bc69041ba8076817c8b665314 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:51.536470 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-505d70e3-983b-40ea-bde1-0cfa0988b007 admin admin] The token cache is being invalidate because project 4b32888ec9554d878c5393719419702a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:20:51.545906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-505d70e3-983b-40ea-bde1-0cfa0988b007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.545906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-505d70e3-983b-40ea-bde1-0cfa0988b007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.545906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1037/2040] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:20:51 2022] DELETE /identity/v3/projects/4b32888ec9554d878c5393719419702a => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:20:51.549604 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e390be-b4cd-47e9-bc8f-087e434c8bcf admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:51.551760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-91e390be-b4cd-47e9-bc8f-087e434c8bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:51.567829 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:51.617058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e390be-b4cd-47e9-bc8f-087e434c8bcf admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:51.621724 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcce3f2-184d-41ec-b41c-d10a2db457c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:51.622379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcce3f2-184d-41ec-b41c-d10a2db457c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:51.623808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcce3f2-184d-41ec-b41c-d10a2db457c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:51.624151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcce3f2-184d-41ec-b41c-d10a2db457c4 admin admin] PATH_INFO: `/v3/users/9e71e24b2490474d8be9abaef687e732` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:51.625370 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcce3f2-184d-41ec-b41c-d10a2db457c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e71e24b2490474d8be9abaef687e732)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:51.627953 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcce3f2-184d-41ec-b41c-d10a2db457c4 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:51.831819 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4dcce3f2-184d-41ec-b41c-d10a2db457c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e71e24b2490474d8be9abaef687e732'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.832472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4dcce3f2-184d-41ec-b41c-d10a2db457c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e71e24b2490474d8be9abaef687e732'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.967059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4dcce3f2-184d-41ec-b41c-d10a2db457c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e71e24b2490474d8be9abaef687e732'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.975041 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d70e3-983b-40ea-bde1-0cfa0988b007 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:51.976693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-505d70e3-983b-40ea-bde1-0cfa0988b007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:51.980879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4dcce3f2-184d-41ec-b41c-d10a2db457c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e71e24b2490474d8be9abaef687e732'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:51.994378 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4dcce3f2-184d-41ec-b41c-d10a2db457c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e71e24b2490474d8be9abaef687e732'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:52.010397 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1004/2041] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:20:51 2022] DELETE /identity/v3/users/9e71e24b2490474d8be9abaef687e732 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:52.061562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d70e3-983b-40ea-bde1-0cfa0988b007 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:52.113546 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:52.119411 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794ba37-8637-4d47-9e63-8c889524c8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:52.120044 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794ba37-8637-4d47-9e63-8c889524c8f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:52.120328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794ba37-8637-4d47-9e63-8c889524c8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:52.120605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794ba37-8637-4d47-9e63-8c889524c8f1 admin admin] PATH_INFO: `/v3/projects/54c5d5a82f8d46749873ea5a7837e128` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:52.121873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794ba37-8637-4d47-9e63-8c889524c8f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54c5d5a82f8d46749873ea5a7837e128)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:52.123802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1794ba37-8637-4d47-9e63-8c889524c8f1 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:52.281363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1794ba37-8637-4d47-9e63-8c889524c8f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54c5d5a82f8d46749873ea5a7837e128', 'request_id': 'req-1794ba37-8637-4d47-9e63-8c889524c8f1'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:52.288858 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1794ba37-8637-4d47-9e63-8c889524c8f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54c5d5a82f8d46749873ea5a7837e128', 'request_id': 'req-1794ba37-8637-4d47-9e63-8c889524c8f1'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:52.297021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1794ba37-8637-4d47-9e63-8c889524c8f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54c5d5a82f8d46749873ea5a7837e128', 'request_id': 'req-1794ba37-8637-4d47-9e63-8c889524c8f1'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:52.418052 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1794ba37-8637-4d47-9e63-8c889524c8f1 admin admin] The token cache is being invalidate because project 54c5d5a82f8d46749873ea5a7837e128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:20:52.418750 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1794ba37-8637-4d47-9e63-8c889524c8f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:52.419153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1794ba37-8637-4d47-9e63-8c889524c8f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:52.421230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1038/2042] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:20:51 2022] DELETE /identity/v3/projects/54c5d5a82f8d46749873ea5a7837e128 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:20:52.426253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bac4a-8e32-44f5-a0eb-5864a79d4818 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:52.426589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bac4a-8e32-44f5-a0eb-5864a79d4818 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:52.426907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bac4a-8e32-44f5-a0eb-5864a79d4818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:52.447651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-939bac4a-8e32-44f5-a0eb-5864a79d4818 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:52.448914 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-939bac4a-8e32-44f5-a0eb-5864a79d4818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:52.500077 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794ba37-8637-4d47-9e63-8c889524c8f1 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:52.502358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-1794ba37-8637-4d47-9e63-8c889524c8f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:52.516879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1005/2043] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:52.526195 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353f598-87de-4c94-99bb-db80670add19 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:52.526510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353f598-87de-4c94-99bb-db80670add19 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:52.526757 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353f598-87de-4c94-99bb-db80670add19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:52.549409 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-6353f598-87de-4c94-99bb-db80670add19 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:52.550569 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6353f598-87de-4c94-99bb-db80670add19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:52.562392 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1006/2044] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:52.563521 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353f598-87de-4c94-99bb-db80670add19 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:52.574310 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353f598-87de-4c94-99bb-db80670add19 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:52.580463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574317d5-42a2-46a2-ade8-5afdfee24cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:52.581099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574317d5-42a2-46a2-ade8-5afdfee24cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:52.581401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574317d5-42a2-46a2-ade8-5afdfee24cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:52.581667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574317d5-42a2-46a2-ade8-5afdfee24cb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:52.582340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574317d5-42a2-46a2-ade8-5afdfee24cb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:20:52.582707 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574317d5-42a2-46a2-ade8-5afdfee24cb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:52.585123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574317d5-42a2-46a2-ade8-5afdfee24cb3 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:52.593868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794ba37-8637-4d47-9e63-8c889524c8f1 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:52.599365 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1007/2045] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:20:52.601131 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36b41d-158e-4dcd-b209-8770d1caa7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:52.601971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36b41d-158e-4dcd-b209-8770d1caa7b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:52.602469 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36b41d-158e-4dcd-b209-8770d1caa7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:52.602885 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36b41d-158e-4dcd-b209-8770d1caa7b3 admin admin] PATH_INFO: `/v3/projects/1a5efafea5a1422f9672ef3c5ed38299` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:52.604512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36b41d-158e-4dcd-b209-8770d1caa7b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a5efafea5a1422f9672ef3c5ed38299)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:52.606478 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36b41d-158e-4dcd-b209-8770d1caa7b3 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:52.614626 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574317d5-42a2-46a2-ade8-5afdfee24cb3 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:52.627556 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574317d5-42a2-46a2-ade8-5afdfee24cb3 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:52.635344 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a7f0f-78c3-40e1-b41e-85863724bc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:52.640790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a7f0f-78c3-40e1-b41e-85863724bc89 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:52.640790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a7f0f-78c3-40e1-b41e-85863724bc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:52.640790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a7f0f-78c3-40e1-b41e-85863724bc89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:52.640790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a7f0f-78c3-40e1-b41e-85863724bc89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:52.641413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a7f0f-78c3-40e1-b41e-85863724bc89 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:52.662051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:52.734656 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4f36b41d-158e-4dcd-b209-8770d1caa7b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a5efafea5a1422f9672ef3c5ed38299', 'request_id': 'req-4f36b41d-158e-4dcd-b209-8770d1caa7b3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:52.740590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4f36b41d-158e-4dcd-b209-8770d1caa7b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a5efafea5a1422f9672ef3c5ed38299', 'request_id': 'req-4f36b41d-158e-4dcd-b209-8770d1caa7b3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:52.757236 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4f36b41d-158e-4dcd-b209-8770d1caa7b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a5efafea5a1422f9672ef3c5ed38299', 'request_id': 'req-4f36b41d-158e-4dcd-b209-8770d1caa7b3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:52.769684 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1008/2046] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:20:52 2022] POST /identity/v3/projects => generated 371 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:52.775033 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a7f0f-78c3-40e1-b41e-85863724bc89 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:52.845618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4f36b41d-158e-4dcd-b209-8770d1caa7b3 admin admin] The token cache is being invalidate because project 1a5efafea5a1422f9672ef3c5ed38299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:20:52.845977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4f36b41d-158e-4dcd-b209-8770d1caa7b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:52.847339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4f36b41d-158e-4dcd-b209-8770d1caa7b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:52.853773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1039/2047] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:20:52 2022] DELETE /identity/v3/projects/1a5efafea5a1422f9672ef3c5ed38299 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:20:52.855927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154beb5-4526-4505-801e-9a295bf441e4 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:52.856333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154beb5-4526-4505-801e-9a295bf441e4 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:52.856770 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154beb5-4526-4505-801e-9a295bf441e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:52.877945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a7f0f-78c3-40e1-b41e-85863724bc89 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:52.881219 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-0154beb5-4526-4505-801e-9a295bf441e4 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:52.882671 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0154beb5-4526-4505-801e-9a295bf441e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:52.883272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1b816-be03-4881-9d87-6f5635a71b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:52.883904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1b816-be03-4881-9d87-6f5635a71b77 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:52.884196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1b816-be03-4881-9d87-6f5635a71b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:52.884628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1b816-be03-4881-9d87-6f5635a71b77 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:52.885312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1b816-be03-4881-9d87-6f5635a71b77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:52.887248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1b816-be03-4881-9d87-6f5635a71b77 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:52.892119 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1040/2048] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:52.900182 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee96df-74e0-4ddf-b12f-02d4f730e0dd None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:52.900572 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee96df-74e0-4ddf-b12f-02d4f730e0dd None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:52.900955 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee96df-74e0-4ddf-b12f-02d4f730e0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:52.934961 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-0eee96df-74e0-4ddf-b12f-02d4f730e0dd None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:52.936220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0eee96df-74e0-4ddf-b12f-02d4f730e0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:52.954794 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1041/2049] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:52.958854 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee96df-74e0-4ddf-b12f-02d4f730e0dd None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:52.964732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:52.969011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee96df-74e0-4ddf-b12f-02d4f730e0dd None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:52.974515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48242fa6-5101-41eb-ac73-092b45a59554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:52.977179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48242fa6-5101-41eb-ac73-092b45a59554 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:52.977179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48242fa6-5101-41eb-ac73-092b45a59554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:52.977179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48242fa6-5101-41eb-ac73-092b45a59554 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:52.977179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48242fa6-5101-41eb-ac73-092b45a59554 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:20:52.977179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48242fa6-5101-41eb-ac73-092b45a59554 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:52.978885 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48242fa6-5101-41eb-ac73-092b45a59554 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:52.979628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1009/2050] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:20:52 2022] POST /identity/v3/users => generated 286 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:52.983725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1b816-be03-4881-9d87-6f5635a71b77 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:52.986116 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-08f1b816-be03-4881-9d87-6f5635a71b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:52.991904 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1042/2051] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:20:52.998423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48242fa6-5101-41eb-ac73-092b45a59554 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:53.003880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1b816-be03-4881-9d87-6f5635a71b77 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:53.009480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48242fa6-5101-41eb-ac73-092b45a59554 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:53.009903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541aa89-c637-4758-b610-f55d4aa74585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:53.010627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541aa89-c637-4758-b610-f55d4aa74585 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:53.010965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541aa89-c637-4758-b610-f55d4aa74585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:53.011288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541aa89-c637-4758-b610-f55d4aa74585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:53.012096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541aa89-c637-4758-b610-f55d4aa74585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:53.013708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541aa89-c637-4758-b610-f55d4aa74585 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:53.015540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbf0ee-7ba9-4a2e-8605-9b95b5d2f97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:53.016145 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbf0ee-7ba9-4a2e-8605-9b95b5d2f97a admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:53.016463 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbf0ee-7ba9-4a2e-8605-9b95b5d2f97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:53.016879 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbf0ee-7ba9-4a2e-8605-9b95b5d2f97a admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:53.017511 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbf0ee-7ba9-4a2e-8605-9b95b5d2f97a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:53.019648 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbf0ee-7ba9-4a2e-8605-9b95b5d2f97a admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:53.024812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1010/2052] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:20:53.029234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541aa89-c637-4758-b610-f55d4aa74585 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:53.040557 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541aa89-c637-4758-b610-f55d4aa74585 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:53.045996 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e379f90-4ab8-4dec-af73-e9c1ad2a34e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:53.046601 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e379f90-4ab8-4dec-af73-e9c1ad2a34e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:53.046877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e379f90-4ab8-4dec-af73-e9c1ad2a34e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:53.047154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e379f90-4ab8-4dec-af73-e9c1ad2a34e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:53.047807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e379f90-4ab8-4dec-af73-e9c1ad2a34e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:53.050338 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e379f90-4ab8-4dec-af73-e9c1ad2a34e2 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:53.077188 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1011/2053] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:53.081143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e379f90-4ab8-4dec-af73-e9c1ad2a34e2 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:53.091645 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e379f90-4ab8-4dec-af73-e9c1ad2a34e2 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:53.097034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05122408-d77c-4268-acbc-3f80c6188dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:53.097613 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05122408-d77c-4268-acbc-3f80c6188dca admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:53.097900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05122408-d77c-4268-acbc-3f80c6188dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:53.098174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05122408-d77c-4268-acbc-3f80c6188dca admin admin] PATH_INFO: `/v3/projects/8164da757959471cbbfb3b51af1edebd/users/011f5d15687d4a55ac595c8f067d6f1d/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:53.112603 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1043/2054] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:20:52 2022] POST /identity/v3/projects => generated 399 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:53.114460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05122408-d77c-4268-acbc-3f80c6188dca admin admin] RBAC: Authorizing `identity:create_grant(project_id=8164da757959471cbbfb3b51af1edebd, user_id=011f5d15687d4a55ac595c8f067d6f1d, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:53.116806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbf0ee-7ba9-4a2e-8605-9b95b5d2f97a admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:53.117448 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05122408-d77c-4268-acbc-3f80c6188dca admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:53.182502 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbf0ee-7ba9-4a2e-8605-9b95b5d2f97a admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:53.187904 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69548d64-7140-4716-96c1-aa68f522a5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:53.189416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69548d64-7140-4716-96c1-aa68f522a5cc admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:53.190881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69548d64-7140-4716-96c1-aa68f522a5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:53.191551 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69548d64-7140-4716-96c1-aa68f522a5cc admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:53.192555 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69548d64-7140-4716-96c1-aa68f522a5cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:53.199103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69548d64-7140-4716-96c1-aa68f522a5cc admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:53.209744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1012/2055] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:20:53 2022] PUT /identity/v3/projects/8164da757959471cbbfb3b51af1edebd/users/011f5d15687d4a55ac595c8f067d6f1d/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:53.214268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10a9cc-6162-41fd-a122-3fe80b5228ec None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:53.214637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10a9cc-6162-41fd-a122-3fe80b5228ec None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:53.215321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10a9cc-6162-41fd-a122-3fe80b5228ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:53.241199 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-8c10a9cc-6162-41fd-a122-3fe80b5228ec None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:53.242454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8c10a9cc-6162-41fd-a122-3fe80b5228ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:53.294155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1044/2056] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:20:53 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:20:53.307650 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69548d64-7140-4716-96c1-aa68f522a5cc admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:53.322413 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1013/2057] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:53.326712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69548d64-7140-4716-96c1-aa68f522a5cc admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:53.333994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03d796-d451-4aac-aee3-8e8b3d683f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:53.334158 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10a9cc-6162-41fd-a122-3fe80b5228ec None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:53.334610 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03d796-d451-4aac-aee3-8e8b3d683f19 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:53.334927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03d796-d451-4aac-aee3-8e8b3d683f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:53.335196 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03d796-d451-4aac-aee3-8e8b3d683f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:53.335853 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03d796-d451-4aac-aee3-8e8b3d683f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:53.337318 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8c10a9cc-6162-41fd-a122-3fe80b5228ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:53.337515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03d796-d451-4aac-aee3-8e8b3d683f19 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:53.348486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1045/2058] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:53.352969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03d796-d451-4aac-aee3-8e8b3d683f19 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:53.365497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03d796-d451-4aac-aee3-8e8b3d683f19 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:53.371629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eac464-4794-422c-b4d4-766a69415d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:53.371629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eac464-4794-422c-b4d4-766a69415d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:53.372129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eac464-4794-422c-b4d4-766a69415d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:53.373231 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eac464-4794-422c-b4d4-766a69415d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:53.373895 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eac464-4794-422c-b4d4-766a69415d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:53.375501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eac464-4794-422c-b4d4-766a69415d83 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:53.399919 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1046/2059] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:53.401309 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10a9cc-6162-41fd-a122-3fe80b5228ec None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:53.401923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eac464-4794-422c-b4d4-766a69415d83 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:53.405248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65995b0d-ed3e-41f1-9eef-f96e26168eb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:53.405864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65995b0d-ed3e-41f1-9eef-f96e26168eb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:53.406144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65995b0d-ed3e-41f1-9eef-f96e26168eb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:53.406415 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65995b0d-ed3e-41f1-9eef-f96e26168eb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:53.414273 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eac464-4794-422c-b4d4-766a69415d83 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:53.419962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0e39d-fb08-476e-b834-d219112787e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:53.420604 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0e39d-fb08-476e-b834-d219112787e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:53.421179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65995b0d-ed3e-41f1-9eef-f96e26168eb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:53.421347 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0e39d-fb08-476e-b834-d219112787e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:53.421868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0e39d-fb08-476e-b834-d219112787e2 admin admin] PATH_INFO: `/v3/projects/0d68e0d4ada74293b9e22d2c90016929/users/a8bebaf99d494f2b9b42d72c0bf76ae7/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:53.422904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65995b0d-ed3e-41f1-9eef-f96e26168eb1 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:53.438207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1014/2060] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:20:53.440406 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0e39d-fb08-476e-b834-d219112787e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d68e0d4ada74293b9e22d2c90016929, user_id=a8bebaf99d494f2b9b42d72c0bf76ae7, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:53.443212 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0e39d-fb08-476e-b834-d219112787e2 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:53.558969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1047/2061] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:20:53 2022] PUT /identity/v3/projects/0d68e0d4ada74293b9e22d2c90016929/users/a8bebaf99d494f2b9b42d72c0bf76ae7/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:20:53.570395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bde4e-2117-4bd3-a2cd-5fc636ecb672 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:53.570790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bde4e-2117-4bd3-a2cd-5fc636ecb672 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:53.570963 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bde4e-2117-4bd3-a2cd-5fc636ecb672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:53.600394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-1b4bde4e-2117-4bd3-a2cd-5fc636ecb672 None None] MFA Rules not processed for user `a8bebaf99d494f2b9b42d72c0bf76ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:53.601582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1b4bde4e-2117-4bd3-a2cd-5fc636ecb672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:53.668763 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1015/2062] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:53 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:53.673434 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0e39d-fb08-476e-b834-d219112787e2 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:53.734133 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0e39d-fb08-476e-b834-d219112787e2 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:53.738751 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be326f2-7a63-4be9-8e4a-ab3bb9dcd025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:53.739344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be326f2-7a63-4be9-8e4a-ab3bb9dcd025 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:53.739687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be326f2-7a63-4be9-8e4a-ab3bb9dcd025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:53.739966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be326f2-7a63-4be9-8e4a-ab3bb9dcd025 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:53.740616 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be326f2-7a63-4be9-8e4a-ab3bb9dcd025 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:53.742629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be326f2-7a63-4be9-8e4a-ab3bb9dcd025 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:53.847214 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1048/2063] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:20:53 2022] POST /identity/v3/projects => generated 399 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:53.851402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bde4e-2117-4bd3-a2cd-5fc636ecb672 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:53.862183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bde4e-2117-4bd3-a2cd-5fc636ecb672 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:53.897070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:53.925076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383184f-239b-4b9c-bcec-89735e59c95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:53.930664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383184f-239b-4b9c-bcec-89735e59c95c admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:53.930664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383184f-239b-4b9c-bcec-89735e59c95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:53.930664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383184f-239b-4b9c-bcec-89735e59c95c admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:53.930664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383184f-239b-4b9c-bcec-89735e59c95c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:53.930664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383184f-239b-4b9c-bcec-89735e59c95c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:54.013100 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:54.019138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1016/2064] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:20:53 2022] POST /identity/v3/users => generated 306 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:54.023438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be326f2-7a63-4be9-8e4a-ab3bb9dcd025 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:54.034839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be326f2-7a63-4be9-8e4a-ab3bb9dcd025 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:54.039966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcf5b1-bdd0-48b1-8757-bb704a0e40b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:54.040560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcf5b1-bdd0-48b1-8757-bb704a0e40b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:54.040881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcf5b1-bdd0-48b1-8757-bb704a0e40b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:54.041164 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcf5b1-bdd0-48b1-8757-bb704a0e40b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:54.041819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcf5b1-bdd0-48b1-8757-bb704a0e40b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:54.043443 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcf5b1-bdd0-48b1-8757-bb704a0e40b2 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:54.056505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1049/2065] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:20:54.062386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383184f-239b-4b9c-bcec-89735e59c95c admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:54.073972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383184f-239b-4b9c-bcec-89735e59c95c admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:54.080072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208393f3-d1da-40ef-976a-9ccb63a00585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:54.080228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208393f3-d1da-40ef-976a-9ccb63a00585 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:54.080436 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208393f3-d1da-40ef-976a-9ccb63a00585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:54.080769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208393f3-d1da-40ef-976a-9ccb63a00585 admin admin] PATH_INFO: `/v3/projects/03a99843c0a64b96849376e8cb2478ca/users/08440385227847f69c4842973b8cb697/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:54.097154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208393f3-d1da-40ef-976a-9ccb63a00585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03a99843c0a64b96849376e8cb2478ca, user_id=08440385227847f69c4842973b8cb697, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:54.099932 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208393f3-d1da-40ef-976a-9ccb63a00585 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:54.237398 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1017/2066] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:20:54 2022] PUT /identity/v3/projects/03a99843c0a64b96849376e8cb2478ca/users/08440385227847f69c4842973b8cb697/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:54.241592 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcf5b1-bdd0-48b1-8757-bb704a0e40b2 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:54.254193 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcf5b1-bdd0-48b1-8757-bb704a0e40b2 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:54.318994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db38750-e544-46a5-bc71-f4b310f7efcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:54.319705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db38750-e544-46a5-bc71-f4b310f7efcd admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:54.320314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db38750-e544-46a5-bc71-f4b310f7efcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:54.320594 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db38750-e544-46a5-bc71-f4b310f7efcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:54.321437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db38750-e544-46a5-bc71-f4b310f7efcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:54.323027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db38750-e544-46a5-bc71-f4b310f7efcd admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:54.334660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1050/2067] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:54.339482 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208393f3-d1da-40ef-976a-9ccb63a00585 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:54.354009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208393f3-d1da-40ef-976a-9ccb63a00585 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:54.359523 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b626512-4688-4c38-8ec6-15787693dbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:54.374429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b626512-4688-4c38-8ec6-15787693dbcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:54.377915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b626512-4688-4c38-8ec6-15787693dbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:54.378296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b626512-4688-4c38-8ec6-15787693dbcd admin admin] PATH_INFO: `/v3/projects/03a99843c0a64b96849376e8cb2478ca/users/08440385227847f69c4842973b8cb697/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:54.385279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b626512-4688-4c38-8ec6-15787693dbcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=03a99843c0a64b96849376e8cb2478ca, user_id=08440385227847f69c4842973b8cb697, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:54.389695 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b626512-4688-4c38-8ec6-15787693dbcd admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:54.414947 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1018/2068] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:20:54 2022] PUT /identity/v3/projects/03a99843c0a64b96849376e8cb2478ca/users/08440385227847f69c4842973b8cb697/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:54.427524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb946fe6-6fb6-4623-a0db-9808665da50c None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:54.427994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb946fe6-6fb6-4623-a0db-9808665da50c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:54.428427 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb946fe6-6fb6-4623-a0db-9808665da50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:54.469899 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-cb946fe6-6fb6-4623-a0db-9808665da50c None None] MFA Rules not processed for user `08440385227847f69c4842973b8cb697`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:54.471929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-cb946fe6-6fb6-4623-a0db-9808665da50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:54.512826 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b626512-4688-4c38-8ec6-15787693dbcd admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:54.514698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1b626512-4688-4c38-8ec6-15787693dbcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:54.538308 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:54.547694 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1051/2069] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:54 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:54.559949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb946fe6-6fb6-4623-a0db-9808665da50c None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:54.561362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-cb946fe6-6fb6-4623-a0db-9808665da50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:54.598254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b626512-4688-4c38-8ec6-15787693dbcd admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:54.605262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fb134-a958-4808-a14c-67d3c50da0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:54.605898 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fb134-a958-4808-a14c-67d3c50da0a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:54.606179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fb134-a958-4808-a14c-67d3c50da0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:54.606454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fb134-a958-4808-a14c-67d3c50da0a1 admin admin] PATH_INFO: `/v3/users/cc511731cfaf4836b0671146dd4e71bf` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:54.608607 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fb134-a958-4808-a14c-67d3c50da0a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc511731cfaf4836b0671146dd4e71bf)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:54.611296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fb134-a958-4808-a14c-67d3c50da0a1 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:54.634265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb946fe6-6fb6-4623-a0db-9808665da50c None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:54.638318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd82d24-bb8a-404e-b451-8a2c29d4acb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:54.639089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd82d24-bb8a-404e-b451-8a2c29d4acb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:54.639377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd82d24-bb8a-404e-b451-8a2c29d4acb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:54.639654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd82d24-bb8a-404e-b451-8a2c29d4acb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:54.650686 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd82d24-bb8a-404e-b451-8a2c29d4acb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:54.652148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd82d24-bb8a-404e-b451-8a2c29d4acb1 service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:54.661926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1052/2070] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:54 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:20:54.905683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-064fb134-a958-4808-a14c-67d3c50da0a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc511731cfaf4836b0671146dd4e71bf'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:54.906372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-064fb134-a958-4808-a14c-67d3c50da0a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc511731cfaf4836b0671146dd4e71bf'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:54.932055 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26804a8d-1e63-4d2a-88d8-b5f6f37550f1 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:54.932240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26804a8d-1e63-4d2a-88d8-b5f6f37550f1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:54.932520 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26804a8d-1e63-4d2a-88d8-b5f6f37550f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:54.953111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:54.960524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-26804a8d-1e63-4d2a-88d8-b5f6f37550f1 None None] MFA Rules not processed for user `011f5d15687d4a55ac595c8f067d6f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:54.961759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-26804a8d-1e63-4d2a-88d8-b5f6f37550f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:54.966628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-064fb134-a958-4808-a14c-67d3c50da0a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc511731cfaf4836b0671146dd4e71bf'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:54.980304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-064fb134-a958-4808-a14c-67d3c50da0a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc511731cfaf4836b0671146dd4e71bf'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:54.988589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-064fb134-a958-4808-a14c-67d3c50da0a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc511731cfaf4836b0671146dd4e71bf'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:54.995141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1019/2071] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:20:54 2022] DELETE /identity/v3/users/cc511731cfaf4836b0671146dd4e71bf => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:20:54.998978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fb134-a958-4808-a14c-67d3c50da0a1 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:55.000382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-064fb134-a958-4808-a14c-67d3c50da0a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:55.039228 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1053/2072] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:55.040705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f335c-6b12-4288-9e17-2702ed644ae8 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:55.040997 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f335c-6b12-4288-9e17-2702ed644ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:55.041275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f335c-6b12-4288-9e17-2702ed644ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:55.061043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-890f335c-6b12-4288-9e17-2702ed644ae8 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:55.062294 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-890f335c-6b12-4288-9e17-2702ed644ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:55.067067 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fb134-a958-4808-a14c-67d3c50da0a1 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:55.070824 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6f15e-9287-46fe-8132-fc5492721835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:55.071428 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6f15e-9287-46fe-8132-fc5492721835 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:55.071713 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6f15e-9287-46fe-8132-fc5492721835 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:55.071989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6f15e-9287-46fe-8132-fc5492721835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:55.073845 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a7c6f15e-9287-46fe-8132-fc5492721835 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:55.122310 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1054/2073] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:55.123270 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f335c-6b12-4288-9e17-2702ed644ae8 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:55.124536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-890f335c-6b12-4288-9e17-2702ed644ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:55.136985 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f335c-6b12-4288-9e17-2702ed644ae8 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:55.139824 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c6f15e-9287-46fe-8132-fc5492721835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:55.140654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68f19b-053e-4c1a-aa14-5a65361953fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:55.141301 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68f19b-053e-4c1a-aa14-5a65361953fb service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:55.141412 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c6f15e-9287-46fe-8132-fc5492721835 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:55.142112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68f19b-053e-4c1a-aa14-5a65361953fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:55.142422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68f19b-053e-4c1a-aa14-5a65361953fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:55.151734 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1020/2074] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:20:55.152364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6f15e-9287-46fe-8132-fc5492721835 service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:55.153880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a7c6f15e-9287-46fe-8132-fc5492721835 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:55.162278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:55.210961 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68f19b-053e-4c1a-aa14-5a65361953fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:55.212403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68f19b-053e-4c1a-aa14-5a65361953fb service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:55.219402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6f15e-9287-46fe-8132-fc5492721835 service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:55.224335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1eb8d-6702-4fdf-b80c-4cb76180407c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:55.224979 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1eb8d-6702-4fdf-b80c-4cb76180407c service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:55.225396 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1eb8d-6702-4fdf-b80c-4cb76180407c service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:55.225758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1eb8d-6702-4fdf-b80c-4cb76180407c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:55.233965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1055/2075] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:20:55.239106 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1eb8d-6702-4fdf-b80c-4cb76180407c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:55.240823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1eb8d-6702-4fdf-b80c-4cb76180407c service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:55.251588 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1021/2076] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:20:55.801712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68f19b-053e-4c1a-aa14-5a65361953fb service cinder] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:55.803195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-da68f19b-053e-4c1a-aa14-5a65361953fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:55.830180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68f19b-053e-4c1a-aa14-5a65361953fb service cinder] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:55.857143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9b139-60a8-4135-bed8-703d93a09b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:55.857804 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9b139-60a8-4135-bed8-703d93a09b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:55.858092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9b139-60a8-4135-bed8-703d93a09b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:55.858376 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9b139-60a8-4135-bed8-703d93a09b9b admin admin] PATH_INFO: `/v3/projects/39e260f3108d4b9f85088dfd7fbf460d` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:55.862295 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9b139-60a8-4135-bed8-703d93a09b9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=39e260f3108d4b9f85088dfd7fbf460d)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:55.864386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9b139-60a8-4135-bed8-703d93a09b9b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:55.996962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:56.007064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d8d9b139-60a8-4135-bed8-703d93a09b9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39e260f3108d4b9f85088dfd7fbf460d', 'request_id': 'req-d8d9b139-60a8-4135-bed8-703d93a09b9b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:56.013013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d8d9b139-60a8-4135-bed8-703d93a09b9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39e260f3108d4b9f85088dfd7fbf460d', 'request_id': 'req-d8d9b139-60a8-4135-bed8-703d93a09b9b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:56.020252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d8d9b139-60a8-4135-bed8-703d93a09b9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39e260f3108d4b9f85088dfd7fbf460d', 'request_id': 'req-d8d9b139-60a8-4135-bed8-703d93a09b9b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:56.106646 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d8d9b139-60a8-4135-bed8-703d93a09b9b admin admin] The token cache is being invalidate because project 39e260f3108d4b9f85088dfd7fbf460d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:20:56.106939 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d8d9b139-60a8-4135-bed8-703d93a09b9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:56.107585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d8d9b139-60a8-4135-bed8-703d93a09b9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:20:56.109169 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1056/2077] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:20:55 2022] DELETE /identity/v3/projects/39e260f3108d4b9f85088dfd7fbf460d => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:56.114467 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f7a9f-23d4-4015-b107-04e58af41fd1 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:56.114914 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f7a9f-23d4-4015-b107-04e58af41fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:56.115004 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f7a9f-23d4-4015-b107-04e58af41fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:56.138474 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-cf8f7a9f-23d4-4015-b107-04e58af41fd1 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:56.138474 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-cf8f7a9f-23d4-4015-b107-04e58af41fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:56.197348 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1022/2078] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:56.206018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba5965-4ac3-47e9-a1b8-07d1a30e9ce3 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:56.206233 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba5965-4ac3-47e9-a1b8-07d1a30e9ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:56.206506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba5965-4ac3-47e9-a1b8-07d1a30e9ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:56.226745 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-e5ba5965-4ac3-47e9-a1b8-07d1a30e9ce3 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:56.228890 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e5ba5965-4ac3-47e9-a1b8-07d1a30e9ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:56.237216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1057/2079] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:56.241286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f7a9f-23d4-4015-b107-04e58af41fd1 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:56.251052 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f7a9f-23d4-4015-b107-04e58af41fd1 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:56.256535 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab5f6a-3c54-4b38-a0b2-acecf7dcb6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:56.263454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab5f6a-3c54-4b38-a0b2-acecf7dcb6b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:56.263454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab5f6a-3c54-4b38-a0b2-acecf7dcb6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:56.263454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab5f6a-3c54-4b38-a0b2-acecf7dcb6b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:56.263454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab5f6a-3c54-4b38-a0b2-acecf7dcb6b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:20:56.263454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab5f6a-3c54-4b38-a0b2-acecf7dcb6b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:56.263454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab5f6a-3c54-4b38-a0b2-acecf7dcb6b5 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:56.272463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1023/2080] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:20:56.276722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba5965-4ac3-47e9-a1b8-07d1a30e9ce3 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:56.286792 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba5965-4ac3-47e9-a1b8-07d1a30e9ce3 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:56.291605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4f036-2bd7-421d-b8cf-60be0093c76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:56.292198 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4f036-2bd7-421d-b8cf-60be0093c76e admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:56.292514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4f036-2bd7-421d-b8cf-60be0093c76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:56.292830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4f036-2bd7-421d-b8cf-60be0093c76e admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:56.293465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4f036-2bd7-421d-b8cf-60be0093c76e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:56.295412 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4f036-2bd7-421d-b8cf-60be0093c76e admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:56.422631 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1058/2081] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:20:56 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:56.428690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab5f6a-3c54-4b38-a0b2-acecf7dcb6b5 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:56.439922 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab5f6a-3c54-4b38-a0b2-acecf7dcb6b5 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:56.502073 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ef38f-64be-4d9f-8a3e-de3b7e3623cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:56.502608 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ef38f-64be-4d9f-8a3e-de3b7e3623cb admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:56.502888 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ef38f-64be-4d9f-8a3e-de3b7e3623cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:56.503158 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ef38f-64be-4d9f-8a3e-de3b7e3623cb admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:56.503781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ef38f-64be-4d9f-8a3e-de3b7e3623cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:56.505738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ef38f-64be-4d9f-8a3e-de3b7e3623cb admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:56.800469 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1024/2082] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:20:56 2022] POST /identity/v3/users => generated 289 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:56.804423 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4f036-2bd7-421d-b8cf-60be0093c76e admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:56.817541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4f036-2bd7-421d-b8cf-60be0093c76e admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:56.823140 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d5913-190d-4ee9-80ee-8815a39bf76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:56.823780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d5913-190d-4ee9-80ee-8815a39bf76d admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:56.824065 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d5913-190d-4ee9-80ee-8815a39bf76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:56.824343 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d5913-190d-4ee9-80ee-8815a39bf76d admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:56.825029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d5913-190d-4ee9-80ee-8815a39bf76d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:56.826693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d5913-190d-4ee9-80ee-8815a39bf76d admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:56.837451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1059/2083] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:56.841578 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ef38f-64be-4d9f-8a3e-de3b7e3623cb admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:56.852457 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ef38f-64be-4d9f-8a3e-de3b7e3623cb admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:56.857115 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ec53a-4c05-49aa-93ed-58d9c410a707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:56.857718 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ec53a-4c05-49aa-93ed-58d9c410a707 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:56.858004 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ec53a-4c05-49aa-93ed-58d9c410a707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:56.858281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ec53a-4c05-49aa-93ed-58d9c410a707 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:56.858933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ec53a-4c05-49aa-93ed-58d9c410a707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:56.860534 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ec53a-4c05-49aa-93ed-58d9c410a707 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:56.872198 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1025/2084] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:20:56.876409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d5913-190d-4ee9-80ee-8815a39bf76d admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:56.886820 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d5913-190d-4ee9-80ee-8815a39bf76d admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:56.891279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3c35a-4d72-418e-bfac-7688b8fe2cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:56.891916 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3c35a-4d72-418e-bfac-7688b8fe2cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:56.892201 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3c35a-4d72-418e-bfac-7688b8fe2cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:56.892473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3c35a-4d72-418e-bfac-7688b8fe2cfe admin admin] PATH_INFO: `/v3/projects/867a9ed9454d4c6c8bd5b666793bf5c6/users/c13bcda4000d48d6ab9ca7a9bce71fdd/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:56.907547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3c35a-4d72-418e-bfac-7688b8fe2cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=867a9ed9454d4c6c8bd5b666793bf5c6, user_id=c13bcda4000d48d6ab9ca7a9bce71fdd, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:56.910445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3c35a-4d72-418e-bfac-7688b8fe2cfe admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:56.998789 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1060/2085] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:20:56 2022] PUT /identity/v3/projects/867a9ed9454d4c6c8bd5b666793bf5c6/users/c13bcda4000d48d6ab9ca7a9bce71fdd/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:57.009364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26237c-6efe-42c5-9b6d-2bcbc1797a17 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:57.010391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26237c-6efe-42c5-9b6d-2bcbc1797a17 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:57.010676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26237c-6efe-42c5-9b6d-2bcbc1797a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:57.017737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:57.048704 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-fe26237c-6efe-42c5-9b6d-2bcbc1797a17 None None] MFA Rules not processed for user `c13bcda4000d48d6ab9ca7a9bce71fdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:57.051246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-fe26237c-6efe-42c5-9b6d-2bcbc1797a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:57.123371 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1026/2086] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:57.128150 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3c35a-4d72-418e-bfac-7688b8fe2cfe admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:57.190046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3c35a-4d72-418e-bfac-7688b8fe2cfe admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:57.194676 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1b4c0-f49d-45a4-89f1-301e0b5af763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:57.195269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1b4c0-f49d-45a4-89f1-301e0b5af763 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:57.195553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1b4c0-f49d-45a4-89f1-301e0b5af763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:57.195830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1b4c0-f49d-45a4-89f1-301e0b5af763 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:57.196461 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1b4c0-f49d-45a4-89f1-301e0b5af763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:57.198414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1b4c0-f49d-45a4-89f1-301e0b5af763 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:57.334112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1061/2087] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:20:57 2022] POST /identity/v3/projects => generated 375 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:57.338164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26237c-6efe-42c5-9b6d-2bcbc1797a17 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:57.349728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26237c-6efe-42c5-9b6d-2bcbc1797a17 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:57.408380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f8344-9a5a-4813-a356-86f5dcf6d44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:57.409719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f8344-9a5a-4813-a356-86f5dcf6d44f admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:57.410049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f8344-9a5a-4813-a356-86f5dcf6d44f admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:57.410347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f8344-9a5a-4813-a356-86f5dcf6d44f admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:57.411293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f8344-9a5a-4813-a356-86f5dcf6d44f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:57.413385 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f8344-9a5a-4813-a356-86f5dcf6d44f admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:57.501336 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1027/2088] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:20:57 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:57.505523 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1b4c0-f49d-45a4-89f1-301e0b5af763 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:57.516021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1b4c0-f49d-45a4-89f1-301e0b5af763 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:57.521411 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13cb5c-b583-4d0c-8443-1db9664144b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:57.526880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13cb5c-b583-4d0c-8443-1db9664144b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:57.526880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13cb5c-b583-4d0c-8443-1db9664144b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:57.526880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13cb5c-b583-4d0c-8443-1db9664144b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:57.526880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13cb5c-b583-4d0c-8443-1db9664144b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:57.526880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13cb5c-b583-4d0c-8443-1db9664144b9 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:57.537403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1062/2089] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:57.541432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f8344-9a5a-4813-a356-86f5dcf6d44f admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:57.552097 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f8344-9a5a-4813-a356-86f5dcf6d44f admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:57.556743 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd13b00-caa8-4892-bae6-86123efc66a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:57.557331 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd13b00-caa8-4892-bae6-86123efc66a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:57.557610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd13b00-caa8-4892-bae6-86123efc66a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:57.557886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd13b00-caa8-4892-bae6-86123efc66a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:57.558526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd13b00-caa8-4892-bae6-86123efc66a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:57.560258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd13b00-caa8-4892-bae6-86123efc66a3 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:57.570447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1028/2090] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:57.574379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13cb5c-b583-4d0c-8443-1db9664144b9 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:57.585073 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13cb5c-b583-4d0c-8443-1db9664144b9 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:57.589632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba317235-a2b8-45c8-bb00-c10c7ed635ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:57.590268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba317235-a2b8-45c8-bb00-c10c7ed635ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:57.595458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba317235-a2b8-45c8-bb00-c10c7ed635ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:57.595458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba317235-a2b8-45c8-bb00-c10c7ed635ea admin admin] PATH_INFO: `/v3/projects/1f3ea8da62044a55989b651128d063e9/users/a269e68cb5c94ad89b2385e642c25056/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:57.607087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba317235-a2b8-45c8-bb00-c10c7ed635ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f3ea8da62044a55989b651128d063e9, user_id=a269e68cb5c94ad89b2385e642c25056, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:57.609937 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba317235-a2b8-45c8-bb00-c10c7ed635ea admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:57.714574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1063/2091] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:20:57 2022] PUT /identity/v3/projects/1f3ea8da62044a55989b651128d063e9/users/a269e68cb5c94ad89b2385e642c25056/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:57.726439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4aa274-c885-436a-9760-d7f16f281260 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:57.726728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4aa274-c885-436a-9760-d7f16f281260 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:57.727004 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4aa274-c885-436a-9760-d7f16f281260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:57.755081 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-be4aa274-c885-436a-9760-d7f16f281260 None None] MFA Rules not processed for user `a269e68cb5c94ad89b2385e642c25056`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:57.756283 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-be4aa274-c885-436a-9760-d7f16f281260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:57.817457 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:57.820950 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1029/2092] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:57.825562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba317235-a2b8-45c8-bb00-c10c7ed635ea admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:57.836613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:20:57.891778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba317235-a2b8-45c8-bb00-c10c7ed635ea admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:57.897554 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383705e-1dbe-471b-96a2-ab7ef3d239bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:57.898283 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383705e-1dbe-471b-96a2-ab7ef3d239bd admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:57.898672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383705e-1dbe-471b-96a2-ab7ef3d239bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:57.899065 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383705e-1dbe-471b-96a2-ab7ef3d239bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:57.899779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383705e-1dbe-471b-96a2-ab7ef3d239bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:57.901935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383705e-1dbe-471b-96a2-ab7ef3d239bd admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:57.990709 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1064/2093] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:20:57 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:57.994143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4aa274-c885-436a-9760-d7f16f281260 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:58.014417 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4aa274-c885-436a-9760-d7f16f281260 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:58.094321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fd4f4-8707-4295-ae8b-710d221a00ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:58.095915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fd4f4-8707-4295-ae8b-710d221a00ea admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:58.095915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fd4f4-8707-4295-ae8b-710d221a00ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:58.095915 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fd4f4-8707-4295-ae8b-710d221a00ea admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:58.096529 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fd4f4-8707-4295-ae8b-710d221a00ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:58.098894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fd4f4-8707-4295-ae8b-710d221a00ea admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:58.182050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1030/2094] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:20:57 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:20:58.186245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383705e-1dbe-471b-96a2-ab7ef3d239bd admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:58.197535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383705e-1dbe-471b-96a2-ab7ef3d239bd admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:58.202306 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8619f5-705a-46e7-9a53-84e227956e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:58.203037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8619f5-705a-46e7-9a53-84e227956e56 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:58.203333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8619f5-705a-46e7-9a53-84e227956e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:58.203676 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8619f5-705a-46e7-9a53-84e227956e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:58.204394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8619f5-705a-46e7-9a53-84e227956e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:58.206731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8619f5-705a-46e7-9a53-84e227956e56 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:58.218349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1065/2095] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:58.223469 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fd4f4-8707-4295-ae8b-710d221a00ea admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:58.234334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fd4f4-8707-4295-ae8b-710d221a00ea admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:58.239352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edc382-ba2d-416d-b7f9-86402f89c580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:58.240012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edc382-ba2d-416d-b7f9-86402f89c580 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:58.240290 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edc382-ba2d-416d-b7f9-86402f89c580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:58.240558 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edc382-ba2d-416d-b7f9-86402f89c580 admin admin] PATH_INFO: `/v3/projects/e4d0b7698c864f50a4225f411f666ca2/users/4ec908f75f5949cfb124842c077ceded/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:58.256109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44edc382-ba2d-416d-b7f9-86402f89c580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d0b7698c864f50a4225f411f666ca2, user_id=4ec908f75f5949cfb124842c077ceded, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:58.258771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44edc382-ba2d-416d-b7f9-86402f89c580 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:58.330093 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1031/2096] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:20:58 2022] PUT /identity/v3/projects/e4d0b7698c864f50a4225f411f666ca2/users/4ec908f75f5949cfb124842c077ceded/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:58.337853 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8619f5-705a-46e7-9a53-84e227956e56 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:58.348792 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8619f5-705a-46e7-9a53-84e227956e56 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:58.403297 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf24478-c990-48f1-ae8d-0fb10cf2ea41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:58.403892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf24478-c990-48f1-ae8d-0fb10cf2ea41 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:58.404171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf24478-c990-48f1-ae8d-0fb10cf2ea41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:58.404443 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf24478-c990-48f1-ae8d-0fb10cf2ea41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:58.405110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf24478-c990-48f1-ae8d-0fb10cf2ea41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:58.406743 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf24478-c990-48f1-ae8d-0fb10cf2ea41 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:58.417600 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1066/2097] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:20:58.421753 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edc382-ba2d-416d-b7f9-86402f89c580 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:58.431935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edc382-ba2d-416d-b7f9-86402f89c580 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:58.436444 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c646c-99c0-4a17-862a-5f5ea5ed8223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:58.437112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c646c-99c0-4a17-862a-5f5ea5ed8223 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:58.437392 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c646c-99c0-4a17-862a-5f5ea5ed8223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:58.437664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c646c-99c0-4a17-862a-5f5ea5ed8223 admin admin] PATH_INFO: `/v3/projects/e4d0b7698c864f50a4225f411f666ca2/users/4ec908f75f5949cfb124842c077ceded/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:58.440187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c646c-99c0-4a17-862a-5f5ea5ed8223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d0b7698c864f50a4225f411f666ca2, user_id=4ec908f75f5949cfb124842c077ceded, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:58.442884 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c646c-99c0-4a17-862a-5f5ea5ed8223 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:58.453047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1032/2098] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:20:58 2022] PUT /identity/v3/projects/e4d0b7698c864f50a4225f411f666ca2/users/4ec908f75f5949cfb124842c077ceded/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:20:58.464163 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c410bf9-0b01-4fca-99da-5c369b3b1be4 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:58.464459 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c410bf9-0b01-4fca-99da-5c369b3b1be4 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:58.464789 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c410bf9-0b01-4fca-99da-5c369b3b1be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:58.492413 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-8c410bf9-0b01-4fca-99da-5c369b3b1be4 None None] MFA Rules not processed for user `4ec908f75f5949cfb124842c077ceded`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:20:58.493668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-8c410bf9-0b01-4fca-99da-5c369b3b1be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:58.562156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1067/2099] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:20:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:20:58.574150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c646c-99c0-4a17-862a-5f5ea5ed8223 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:20:58.575869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c80c646c-99c0-4a17-862a-5f5ea5ed8223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:20:58.633127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c646c-99c0-4a17-862a-5f5ea5ed8223 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:20:58.636826 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b9033-3924-4496-a3a6-9312e41165fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:20:58.637425 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b9033-3924-4496-a3a6-9312e41165fd service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:20:58.637710 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b9033-3924-4496-a3a6-9312e41165fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:20:58.637984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b9033-3924-4496-a3a6-9312e41165fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:20:58.694591 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b9033-3924-4496-a3a6-9312e41165fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:20:58.696043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b9033-3924-4496-a3a6-9312e41165fd service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:20:58.704750 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1033/2100] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:20:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:21:01.305409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4923367-eacd-4508-9f19-436fab6d29f5 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:01.305994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4923367-eacd-4508-9f19-436fab6d29f5 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:01.305994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4923367-eacd-4508-9f19-436fab6d29f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:01.315525 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:01.326424 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-a4923367-eacd-4508-9f19-436fab6d29f5 None None] MFA Rules not processed for user `011f5d15687d4a55ac595c8f067d6f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:01.327722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a4923367-eacd-4508-9f19-436fab6d29f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:01.393381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1068/2101] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:02.871822 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b9033-3924-4496-a3a6-9312e41165fd service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:02.873302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9d8b9033-3924-4496-a3a6-9312e41165fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:02.884204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b9033-3924-4496-a3a6-9312e41165fd service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:02.887815 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efc4cd-fb31-4272-b461-863a47623a74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:02.888406 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efc4cd-fb31-4272-b461-863a47623a74 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:02.888742 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efc4cd-fb31-4272-b461-863a47623a74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:02.889035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efc4cd-fb31-4272-b461-863a47623a74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:02.893676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:02.947946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efc4cd-fb31-4272-b461-863a47623a74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:02.949452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efc4cd-fb31-4272-b461-863a47623a74 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:02.959099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1034/2102] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:21:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:21:03.626753 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4923367-eacd-4508-9f19-436fab6d29f5 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:03.628319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a4923367-eacd-4508-9f19-436fab6d29f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:03.704257 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4923367-eacd-4508-9f19-436fab6d29f5 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:03.709828 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa2f15-01f7-4c55-8526-3d0a5b29806d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:03.710453 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa2f15-01f7-4c55-8526-3d0a5b29806d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:03.710739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa2f15-01f7-4c55-8526-3d0a5b29806d admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:03.711016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa2f15-01f7-4c55-8526-3d0a5b29806d admin admin] PATH_INFO: `/v3/users/a8bebaf99d494f2b9b42d72c0bf76ae7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:03.712135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa2f15-01f7-4c55-8526-3d0a5b29806d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8bebaf99d494f2b9b42d72c0bf76ae7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:03.714148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa2f15-01f7-4c55-8526-3d0a5b29806d admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:03.898201 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-74aa2f15-01f7-4c55-8526-3d0a5b29806d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8bebaf99d494f2b9b42d72c0bf76ae7'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:03.990842 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-74aa2f15-01f7-4c55-8526-3d0a5b29806d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8bebaf99d494f2b9b42d72c0bf76ae7'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:03.991498 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-74aa2f15-01f7-4c55-8526-3d0a5b29806d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8bebaf99d494f2b9b42d72c0bf76ae7'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:03.999436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-74aa2f15-01f7-4c55-8526-3d0a5b29806d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8bebaf99d494f2b9b42d72c0bf76ae7'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:04.013003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-74aa2f15-01f7-4c55-8526-3d0a5b29806d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8bebaf99d494f2b9b42d72c0bf76ae7'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:04.021981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1069/2103] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:21:03 2022] DELETE /identity/v3/users/a8bebaf99d494f2b9b42d72c0bf76ae7 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:04.026272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571065d-239a-49d6-924d-3f700270e322 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:04.026768 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571065d-239a-49d6-924d-3f700270e322 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:04.026863 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571065d-239a-49d6-924d-3f700270e322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:04.047153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-c571065d-239a-49d6-924d-3f700270e322 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:04.048402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c571065d-239a-49d6-924d-3f700270e322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:04.115310 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1035/2104] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:04.125669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa2f15-01f7-4c55-8526-3d0a5b29806d admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:04.128197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-74aa2f15-01f7-4c55-8526-3d0a5b29806d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:04.172933 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:04.198923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa2f15-01f7-4c55-8526-3d0a5b29806d admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:04.202664 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf0915-f86b-416e-ab31-591a68ace3f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:04.203276 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf0915-f86b-416e-ab31-591a68ace3f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:04.203567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf0915-f86b-416e-ab31-591a68ace3f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:04.203843 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf0915-f86b-416e-ab31-591a68ace3f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:04.213120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf0915-f86b-416e-ab31-591a68ace3f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:04.214581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf0915-f86b-416e-ab31-591a68ace3f3 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:04.224291 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1070/2105] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:21:04.765637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571065d-239a-49d6-924d-3f700270e322 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:04.767110 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c571065d-239a-49d6-924d-3f700270e322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:04.784342 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571065d-239a-49d6-924d-3f700270e322 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:04.789623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:04.790263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:04.790545 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:04.791650 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc admin admin] PATH_INFO: `/v3/projects/0d68e0d4ada74293b9e22d2c90016929` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:04.792078 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d68e0d4ada74293b9e22d2c90016929)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:04.794022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:04.966890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d68e0d4ada74293b9e22d2c90016929', 'request_id': 'req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:04.972552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d68e0d4ada74293b9e22d2c90016929', 'request_id': 'req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:04.980095 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d68e0d4ada74293b9e22d2c90016929', 'request_id': 'req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:05.069201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc admin admin] The token cache is being invalidate because project 0d68e0d4ada74293b9e22d2c90016929 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:21:05.073473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:05.073473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:05.073473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1036/2106] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:21:04 2022] DELETE /identity/v3/projects/0d68e0d4ada74293b9e22d2c90016929 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:05.076511 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf0915-f86b-416e-ab31-591a68ace3f3 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:05.090154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf0915-f86b-416e-ab31-591a68ace3f3 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:05.145318 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:05.146984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:05.173952 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0f0bf-5325-45d8-9a47-47644119533e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:05.175894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0f0bf-5325-45d8-9a47-47644119533e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:05.176455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0f0bf-5325-45d8-9a47-47644119533e admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:05.177018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0f0bf-5325-45d8-9a47-47644119533e admin admin] PATH_INFO: `/v3/users/08440385227847f69c4842973b8cb697` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:05.179931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0f0bf-5325-45d8-9a47-47644119533e admin admin] RBAC: Authorizing `identity:delete_user(user_id=08440385227847f69c4842973b8cb697)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:05.185035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0f0bf-5325-45d8-9a47-47644119533e admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:05.216705 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:05.245913 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7a2d3-b0fc-4b1f-9343-ad728032febc admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:05.249478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f098a3-6d66-4aeb-8ae0-145e4308792e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:05.250085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f098a3-6d66-4aeb-8ae0-145e4308792e service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:05.250362 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f098a3-6d66-4aeb-8ae0-145e4308792e service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:05.250665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f098a3-6d66-4aeb-8ae0-145e4308792e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:05.252462 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d7f098a3-6d66-4aeb-8ae0-145e4308792e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:05.318567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f098a3-6d66-4aeb-8ae0-145e4308792e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:05.319946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f098a3-6d66-4aeb-8ae0-145e4308792e service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:05.330340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1037/2107] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:21:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:21:05.436073 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-38a0f0bf-5325-45d8-9a47-47644119533e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08440385227847f69c4842973b8cb697'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:05.521245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-38a0f0bf-5325-45d8-9a47-47644119533e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08440385227847f69c4842973b8cb697'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:05.523325 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-38a0f0bf-5325-45d8-9a47-47644119533e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08440385227847f69c4842973b8cb697'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:05.537034 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-38a0f0bf-5325-45d8-9a47-47644119533e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08440385227847f69c4842973b8cb697'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:05.553479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-38a0f0bf-5325-45d8-9a47-47644119533e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08440385227847f69c4842973b8cb697'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:05.561522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1071/2108] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:21:05 2022] DELETE /identity/v3/users/08440385227847f69c4842973b8cb697 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:06.502977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f098a3-6d66-4aeb-8ae0-145e4308792e service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:06.502977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d7f098a3-6d66-4aeb-8ae0-145e4308792e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:06.516776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f098a3-6d66-4aeb-8ae0-145e4308792e service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:06.600096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602eeb6e-0eee-4921-bb58-67512d73ec4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:06.600559 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eeb6e-0eee-4921-bb58-67512d73ec4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:06.600900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eeb6e-0eee-4921-bb58-67512d73ec4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:06.601181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eeb6e-0eee-4921-bb58-67512d73ec4c admin admin] PATH_INFO: `/v3/projects/03a99843c0a64b96849376e8cb2478ca` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:06.602411 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602eeb6e-0eee-4921-bb58-67512d73ec4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=03a99843c0a64b96849376e8cb2478ca)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:06.604336 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602eeb6e-0eee-4921-bb58-67512d73ec4c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:06.731660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-602eeb6e-0eee-4921-bb58-67512d73ec4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03a99843c0a64b96849376e8cb2478ca', 'request_id': 'req-602eeb6e-0eee-4921-bb58-67512d73ec4c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:06.739154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-602eeb6e-0eee-4921-bb58-67512d73ec4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03a99843c0a64b96849376e8cb2478ca', 'request_id': 'req-602eeb6e-0eee-4921-bb58-67512d73ec4c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:06.746361 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-602eeb6e-0eee-4921-bb58-67512d73ec4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03a99843c0a64b96849376e8cb2478ca', 'request_id': 'req-602eeb6e-0eee-4921-bb58-67512d73ec4c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:06.826598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-602eeb6e-0eee-4921-bb58-67512d73ec4c admin admin] The token cache is being invalidate because project 03a99843c0a64b96849376e8cb2478ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:21:06.826897 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-602eeb6e-0eee-4921-bb58-67512d73ec4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:06.828200 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-602eeb6e-0eee-4921-bb58-67512d73ec4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:06.833341 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1038/2109] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:21:06 2022] DELETE /identity/v3/projects/03a99843c0a64b96849376e8cb2478ca => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:06.834903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79caf626-c836-42ee-ad91-da52ee1da8e4 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:06.835367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79caf626-c836-42ee-ad91-da52ee1da8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:06.835458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79caf626-c836-42ee-ad91-da52ee1da8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:06.860050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-79caf626-c836-42ee-ad91-da52ee1da8e4 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:06.861388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-79caf626-c836-42ee-ad91-da52ee1da8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:06.912663 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:06.927826 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1072/2110] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:06.936131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f442a-466e-486a-aa73-f549dff52fa8 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:06.936469 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f442a-466e-486a-aa73-f549dff52fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:06.936737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f442a-466e-486a-aa73-f549dff52fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:06.976616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-2e1f442a-466e-486a-aa73-f549dff52fa8 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:06.978155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2e1f442a-466e-486a-aa73-f549dff52fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:06.992588 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1039/2111] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:06.994027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79caf626-c836-42ee-ad91-da52ee1da8e4 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:07.008165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79caf626-c836-42ee-ad91-da52ee1da8e4 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:07.013441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b7444-910b-4ceb-b2ff-6f4f1b51e5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:07.014113 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b7444-910b-4ceb-b2ff-6f4f1b51e5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:07.014524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b7444-910b-4ceb-b2ff-6f4f1b51e5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:07.014757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b7444-910b-4ceb-b2ff-6f4f1b51e5a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:07.016085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b7444-910b-4ceb-b2ff-6f4f1b51e5a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:21:07.016400 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b7444-910b-4ceb-b2ff-6f4f1b51e5a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:07.021261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b7444-910b-4ceb-b2ff-6f4f1b51e5a1 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:07.049020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1073/2112] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:21:07.050469 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f442a-466e-486a-aa73-f549dff52fa8 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:07.060032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:07.069043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f442a-466e-486a-aa73-f549dff52fa8 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:07.075912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf5e2b-dd03-49d5-b86e-25ffc3f3b22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:07.077476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf5e2b-dd03-49d5-b86e-25ffc3f3b22a admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:07.077973 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf5e2b-dd03-49d5-b86e-25ffc3f3b22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:07.078578 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf5e2b-dd03-49d5-b86e-25ffc3f3b22a admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:07.079838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf5e2b-dd03-49d5-b86e-25ffc3f3b22a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:07.083473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf5e2b-dd03-49d5-b86e-25ffc3f3b22a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:07.232026 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1040/2113] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:21:07 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:21:07.232914 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b7444-910b-4ceb-b2ff-6f4f1b51e5a1 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:07.260617 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b7444-910b-4ceb-b2ff-6f4f1b51e5a1 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:07.331798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c792888-2077-4228-825d-3f0179452a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:07.332425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c792888-2077-4228-825d-3f0179452a75 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:07.332824 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c792888-2077-4228-825d-3f0179452a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:07.333213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c792888-2077-4228-825d-3f0179452a75 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:07.333991 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c792888-2077-4228-825d-3f0179452a75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:07.336249 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c792888-2077-4228-825d-3f0179452a75 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:07.464126 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1074/2114] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:21:07 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:21:07.468239 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf5e2b-dd03-49d5-b86e-25ffc3f3b22a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:07.484230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c792888-2077-4228-825d-3f0179452a75 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:07.487860 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0c792888-2077-4228-825d-3f0179452a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:07.492980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf5e2b-dd03-49d5-b86e-25ffc3f3b22a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:07.506123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e24415-acd9-4036-a5e9-597ec1ec7d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:07.507635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e24415-acd9-4036-a5e9-597ec1ec7d0c admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:07.507635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e24415-acd9-4036-a5e9-597ec1ec7d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:07.507635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e24415-acd9-4036-a5e9-597ec1ec7d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:07.507982 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e24415-acd9-4036-a5e9-597ec1ec7d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:07.509712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e24415-acd9-4036-a5e9-597ec1ec7d0c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:07.511571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c792888-2077-4228-825d-3f0179452a75 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:07.522947 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47de1d4-3fbe-48e5-9f3e-010cc1775de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:07.523601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47de1d4-3fbe-48e5-9f3e-010cc1775de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:07.523887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47de1d4-3fbe-48e5-9f3e-010cc1775de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:07.524166 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47de1d4-3fbe-48e5-9f3e-010cc1775de6 admin admin] PATH_INFO: `/v3/users/f48af9d33a614ab4b100cd72848c7c30` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:07.525494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47de1d4-3fbe-48e5-9f3e-010cc1775de6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f48af9d33a614ab4b100cd72848c7c30)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:07.527507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47de1d4-3fbe-48e5-9f3e-010cc1775de6 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:07.531116 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1041/2115] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:21:07.536589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e24415-acd9-4036-a5e9-597ec1ec7d0c admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:07.551197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e24415-acd9-4036-a5e9-597ec1ec7d0c admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:07.559618 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6726dcf-881e-47d1-856e-8018a4f36d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:07.560681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6726dcf-881e-47d1-856e-8018a4f36d89 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:07.561111 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6726dcf-881e-47d1-856e-8018a4f36d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:07.561528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6726dcf-881e-47d1-856e-8018a4f36d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:07.562318 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6726dcf-881e-47d1-856e-8018a4f36d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:07.564468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6726dcf-881e-47d1-856e-8018a4f36d89 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:07.576120 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1042/2116] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:21:07.580297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6726dcf-881e-47d1-856e-8018a4f36d89 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:07.591252 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6726dcf-881e-47d1-856e-8018a4f36d89 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:07.596047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe733e0-b0ee-4c79-a373-09da48dc73b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:07.596923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe733e0-b0ee-4c79-a373-09da48dc73b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:07.597356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe733e0-b0ee-4c79-a373-09da48dc73b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:07.597811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe733e0-b0ee-4c79-a373-09da48dc73b4 admin admin] PATH_INFO: `/v3/projects/b338e860be6b4c1ea00ad1ab95b0317e/users/dd4fa310d8594198a47ee4f374045fe4/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:07.613217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe733e0-b0ee-4c79-a373-09da48dc73b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b338e860be6b4c1ea00ad1ab95b0317e, user_id=dd4fa310d8594198a47ee4f374045fe4, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:07.615856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe733e0-b0ee-4c79-a373-09da48dc73b4 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:07.727250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1043/2117] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:21:07 2022] PUT /identity/v3/projects/b338e860be6b4c1ea00ad1ab95b0317e/users/dd4fa310d8594198a47ee4f374045fe4/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:21:07.737989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fa19b-23f4-4c4e-8f93-2a6e72979f58 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:07.739008 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fa19b-23f4-4c4e-8f93-2a6e72979f58 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:07.739287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fa19b-23f4-4c4e-8f93-2a6e72979f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:07.771579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-e40fa19b-23f4-4c4e-8f93-2a6e72979f58 None None] MFA Rules not processed for user `dd4fa310d8594198a47ee4f374045fe4`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:07.773185 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-e40fa19b-23f4-4c4e-8f93-2a6e72979f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:07.773689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f47de1d4-3fbe-48e5-9f3e-010cc1775de6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f48af9d33a614ab4b100cd72848c7c30'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:07.862543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1044/2118] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:07.867425 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fa19b-23f4-4c4e-8f93-2a6e72979f58 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:07.869145 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-e40fa19b-23f4-4c4e-8f93-2a6e72979f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:07.885958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f47de1d4-3fbe-48e5-9f3e-010cc1775de6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f48af9d33a614ab4b100cd72848c7c30'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:07.890728 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f47de1d4-3fbe-48e5-9f3e-010cc1775de6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f48af9d33a614ab4b100cd72848c7c30'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:07.895539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f47de1d4-3fbe-48e5-9f3e-010cc1775de6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f48af9d33a614ab4b100cd72848c7c30'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:07.909679 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f47de1d4-3fbe-48e5-9f3e-010cc1775de6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f48af9d33a614ab4b100cd72848c7c30'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:07.916282 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1075/2119] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:21:07 2022] DELETE /identity/v3/users/f48af9d33a614ab4b100cd72848c7c30 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:21:07.920307 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb850c0-9870-4a27-af36-9f183ee289d3 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:07.924056 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb850c0-9870-4a27-af36-9f183ee289d3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:07.924056 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb850c0-9870-4a27-af36-9f183ee289d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:07.931750 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fa19b-23f4-4c4e-8f93-2a6e72979f58 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:07.942572 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-4eb850c0-9870-4a27-af36-9f183ee289d3 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:07.943790 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4eb850c0-9870-4a27-af36-9f183ee289d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:07.990499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd3dd2-c91d-448c-bace-b6be6f355f2e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe9f3fec7104a289e05e88e2d751458', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:07.991109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd3dd2-c91d-448c-bace-b6be6f355f2e service glance] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:07.991389 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd3dd2-c91d-448c-bace-b6be6f355f2e service glance] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:07.991679 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd3dd2-c91d-448c-bace-b6be6f355f2e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:07.993487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-bedd3dd2-c91d-448c-bace-b6be6f355f2e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:08.008372 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1076/2120] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:21:08.038082 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb850c0-9870-4a27-af36-9f183ee289d3 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:08.039570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4eb850c0-9870-4a27-af36-9f183ee289d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:08.093341 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd3dd2-c91d-448c-bace-b6be6f355f2e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:08.095424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd3dd2-c91d-448c-bace-b6be6f355f2e service glance] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:08.107793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1045/2121] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:21:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:21:08.114113 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb850c0-9870-4a27-af36-9f183ee289d3 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:08.120047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95846b9-f528-4584-9318-786855779005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:08.120700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95846b9-f528-4584-9318-786855779005 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:08.121033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95846b9-f528-4584-9318-786855779005 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:08.121883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95846b9-f528-4584-9318-786855779005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:08.138771 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95846b9-f528-4584-9318-786855779005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:08.140735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95846b9-f528-4584-9318-786855779005 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:08.165106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1077/2122] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:21:08.251817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:08.311596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:09.024144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd3dd2-c91d-448c-bace-b6be6f355f2e service glance] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:09.025784 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-bedd3dd2-c91d-448c-bace-b6be6f355f2e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:09.055895 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd3dd2-c91d-448c-bace-b6be6f355f2e service glance] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:09.062985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66452aa6-5b2d-40b4-b087-a26aaa936312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:09.064042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66452aa6-5b2d-40b4-b087-a26aaa936312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:09.064472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66452aa6-5b2d-40b4-b087-a26aaa936312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:09.065016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66452aa6-5b2d-40b4-b087-a26aaa936312 admin admin] PATH_INFO: `/v3/projects/8e68b526cb844c619138688fc68e459f` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:09.066461 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66452aa6-5b2d-40b4-b087-a26aaa936312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e68b526cb844c619138688fc68e459f)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:09.069516 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66452aa6-5b2d-40b4-b087-a26aaa936312 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:09.307828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-66452aa6-5b2d-40b4-b087-a26aaa936312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e68b526cb844c619138688fc68e459f', 'request_id': 'req-66452aa6-5b2d-40b4-b087-a26aaa936312'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:09.319739 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-66452aa6-5b2d-40b4-b087-a26aaa936312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e68b526cb844c619138688fc68e459f', 'request_id': 'req-66452aa6-5b2d-40b4-b087-a26aaa936312'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:09.329636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-66452aa6-5b2d-40b4-b087-a26aaa936312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e68b526cb844c619138688fc68e459f', 'request_id': 'req-66452aa6-5b2d-40b4-b087-a26aaa936312'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:09.401678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-66452aa6-5b2d-40b4-b087-a26aaa936312 admin admin] The token cache is being invalidate because project 8e68b526cb844c619138688fc68e459f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:21:09.402051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-66452aa6-5b2d-40b4-b087-a26aaa936312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:09.402728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-66452aa6-5b2d-40b4-b087-a26aaa936312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:09.404258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1046/2123] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:21:09 2022] DELETE /identity/v3/projects/8e68b526cb844c619138688fc68e459f => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:21:09.410582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25587b-e746-452c-b316-15d14023022b None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:09.411130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25587b-e746-452c-b316-15d14023022b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:09.411223 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25587b-e746-452c-b316-15d14023022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:09.436050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9e25587b-e746-452c-b316-15d14023022b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:09.437299 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9e25587b-e746-452c-b316-15d14023022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:09.506872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1078/2124] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:09.515522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96919b-06cb-4aee-9cf5-935378b1a09c None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:09.515813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96919b-06cb-4aee-9cf5-935378b1a09c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:09.516096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96919b-06cb-4aee-9cf5-935378b1a09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:09.542959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-4e96919b-06cb-4aee-9cf5-935378b1a09c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:09.544271 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4e96919b-06cb-4aee-9cf5-935378b1a09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:09.554066 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1047/2125] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:09.559759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25587b-e746-452c-b316-15d14023022b None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:09.569161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25587b-e746-452c-b316-15d14023022b None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:09.573762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e91e00-f45c-4c09-a2b9-802d9ec891b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:09.574367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e91e00-f45c-4c09-a2b9-802d9ec891b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:09.574644 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e91e00-f45c-4c09-a2b9-802d9ec891b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:09.574923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e91e00-f45c-4c09-a2b9-802d9ec891b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:09.575571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e91e00-f45c-4c09-a2b9-802d9ec891b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:21:09.575873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e91e00-f45c-4c09-a2b9-802d9ec891b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:09.577529 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e91e00-f45c-4c09-a2b9-802d9ec891b8 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:09.590535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1079/2126] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:21:09.595388 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96919b-06cb-4aee-9cf5-935378b1a09c None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:09.605720 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96919b-06cb-4aee-9cf5-935378b1a09c None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:09.611974 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42f772-946e-4d8e-b3a2-1ebe5787cd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:09.612576 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42f772-946e-4d8e-b3a2-1ebe5787cd9a admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:09.613026 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42f772-946e-4d8e-b3a2-1ebe5787cd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:09.613297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42f772-946e-4d8e-b3a2-1ebe5787cd9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:09.613940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42f772-946e-4d8e-b3a2-1ebe5787cd9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:09.615848 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42f772-946e-4d8e-b3a2-1ebe5787cd9a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:09.748224 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1048/2127] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:21:09 2022] POST /identity/v3/projects => generated 371 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:21:09.752294 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e91e00-f45c-4c09-a2b9-802d9ec891b8 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:09.800725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e91e00-f45c-4c09-a2b9-802d9ec891b8 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:09.926673 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b08d50-e5fb-43ad-8e07-d109eb46da77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:09.927293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b08d50-e5fb-43ad-8e07-d109eb46da77 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:09.927574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b08d50-e5fb-43ad-8e07-d109eb46da77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:09.927847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b08d50-e5fb-43ad-8e07-d109eb46da77 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:09.928478 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b08d50-e5fb-43ad-8e07-d109eb46da77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:09.933442 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b08d50-e5fb-43ad-8e07-d109eb46da77 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:10.035201 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:10.038013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1080/2128] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:21:09 2022] POST /identity/v3/users => generated 286 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:21:10.041621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42f772-946e-4d8e-b3a2-1ebe5787cd9a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:10.053095 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42f772-946e-4d8e-b3a2-1ebe5787cd9a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:10.059008 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9c9a1-00a8-4427-ae99-288077419cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:10.059684 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9c9a1-00a8-4427-ae99-288077419cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:10.060099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9c9a1-00a8-4427-ae99-288077419cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:10.060518 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9c9a1-00a8-4427-ae99-288077419cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:10.061225 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9c9a1-00a8-4427-ae99-288077419cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:10.063027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9c9a1-00a8-4427-ae99-288077419cc9 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:10.077064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1049/2129] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:21:10.084624 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b08d50-e5fb-43ad-8e07-d109eb46da77 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:10.095158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b08d50-e5fb-43ad-8e07-d109eb46da77 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:10.101367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63bc39-9cb8-4e85-a9d8-07224171cba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:10.101983 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63bc39-9cb8-4e85-a9d8-07224171cba0 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:10.102528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63bc39-9cb8-4e85-a9d8-07224171cba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:10.102528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63bc39-9cb8-4e85-a9d8-07224171cba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:10.103175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63bc39-9cb8-4e85-a9d8-07224171cba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:10.104881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63bc39-9cb8-4e85-a9d8-07224171cba0 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:10.120975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1081/2130] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:21:10.122820 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9c9a1-00a8-4427-ae99-288077419cc9 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:10.133248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9c9a1-00a8-4427-ae99-288077419cc9 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:10.137857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b75132-f465-4582-bb24-490915043dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:10.138506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b75132-f465-4582-bb24-490915043dbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:10.138790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b75132-f465-4582-bb24-490915043dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:10.139066 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b75132-f465-4582-bb24-490915043dbd admin admin] PATH_INFO: `/v3/projects/61d4fcdc0ddb4a30b0365284ca0e180c/users/b220740b1c204c968864226ae00a2272/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:10.156597 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b75132-f465-4582-bb24-490915043dbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=61d4fcdc0ddb4a30b0365284ca0e180c, user_id=b220740b1c204c968864226ae00a2272, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:10.161307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b75132-f465-4582-bb24-490915043dbd admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:10.259678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1050/2131] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:21:10 2022] PUT /identity/v3/projects/61d4fcdc0ddb4a30b0365284ca0e180c/users/b220740b1c204c968864226ae00a2272/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:10.261224 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a08fcc-b4a4-4c60-be49-ead50cc4f2ea None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:10.261573 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a08fcc-b4a4-4c60-be49-ead50cc4f2ea None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:10.261887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a08fcc-b4a4-4c60-be49-ead50cc4f2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:10.294413 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-72a08fcc-b4a4-4c60-be49-ead50cc4f2ea None None] MFA Rules not processed for user `b220740b1c204c968864226ae00a2272`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:10.296061 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-72a08fcc-b4a4-4c60-be49-ead50cc4f2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:10.457102 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1082/2132] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:10.468820 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b75132-f465-4582-bb24-490915043dbd admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:10.470595 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d8b75132-f465-4582-bb24-490915043dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:10.496552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:10.540492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b75132-f465-4582-bb24-490915043dbd admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:10.547508 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09074e-8975-4824-8927-01d6841cd88e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe9f3fec7104a289e05e88e2d751458', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:10.548115 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09074e-8975-4824-8927-01d6841cd88e service glance] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:10.549323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09074e-8975-4824-8927-01d6841cd88e service glance] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:10.549323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09074e-8975-4824-8927-01d6841cd88e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:10.558088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09074e-8975-4824-8927-01d6841cd88e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:10.559791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09074e-8975-4824-8927-01d6841cd88e service glance] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:10.572505 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1051/2133] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:21:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:21:10.692312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0542592-8502-46ff-a012-77d2524aea42 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:10.692603 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0542592-8502-46ff-a012-77d2524aea42 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:10.692909 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0542592-8502-46ff-a012-77d2524aea42 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:10.694359 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1083/2134] 10.222.0.27 () {54 vars in 921 bytes} [Fri Aug 12 12:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:21:10.728160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4cbd9c-a23c-4185-a506-e363a2e51a44 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:10.728912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4cbd9c-a23c-4185-a506-e363a2e51a44 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:10.728912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4cbd9c-a23c-4185-a506-e363a2e51a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:10.765254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-6c4cbd9c-a23c-4185-a506-e363a2e51a44 None None] MFA Rules not processed for user `5ee8ba21bc184feeaafacab3e69e87bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:10.766623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6c4cbd9c-a23c-4185-a506-e363a2e51a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:10.865219 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0542592-8502-46ff-a012-77d2524aea42 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:10.866981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-c0542592-8502-46ff-a012-77d2524aea42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:10.899597 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1052/2135] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Aug 12 12:21:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:11.000473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0542592-8502-46ff-a012-77d2524aea42 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:11.013426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90efa2-04e3-4c2f-8319-61b9523cf05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:11.014271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90efa2-04e3-4c2f-8319-61b9523cf05a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:11.014716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90efa2-04e3-4c2f-8319-61b9523cf05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:11.015065 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90efa2-04e3-4c2f-8319-61b9523cf05a admin admin] PATH_INFO: `/v3/users/9bc7ffa8e3cf49fe9dcd8471c25be83a` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:11.016546 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90efa2-04e3-4c2f-8319-61b9523cf05a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bc7ffa8e3cf49fe9dcd8471c25be83a)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:11.019606 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90efa2-04e3-4c2f-8319-61b9523cf05a admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:11.176444 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4cbd9c-a23c-4185-a506-e363a2e51a44 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:11.178337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6c4cbd9c-a23c-4185-a506-e363a2e51a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:11.216764 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4cbd9c-a23c-4185-a506-e363a2e51a44 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:11.232283 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06596c6-af12-4bd1-819b-2c158a6275e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ee8ba21bc184feeaafacab3e69e87bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:11.241811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06596c6-af12-4bd1-819b-2c158a6275e7 service placement] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:11.243668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06596c6-af12-4bd1-819b-2c158a6275e7 service placement] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:11.243967 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06596c6-af12-4bd1-819b-2c158a6275e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:11.290927 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06596c6-af12-4bd1-819b-2c158a6275e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:11.291019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1b90efa2-04e3-4c2f-8319-61b9523cf05a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bc7ffa8e3cf49fe9dcd8471c25be83a'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:11.295157 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06596c6-af12-4bd1-819b-2c158a6275e7 service placement] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:11.303061 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1053/2136] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:21:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:21:11.371304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1b90efa2-04e3-4c2f-8319-61b9523cf05a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bc7ffa8e3cf49fe9dcd8471c25be83a'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:11.371940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1b90efa2-04e3-4c2f-8319-61b9523cf05a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bc7ffa8e3cf49fe9dcd8471c25be83a'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:11.380426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1b90efa2-04e3-4c2f-8319-61b9523cf05a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bc7ffa8e3cf49fe9dcd8471c25be83a'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:11.395260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1b90efa2-04e3-4c2f-8319-61b9523cf05a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bc7ffa8e3cf49fe9dcd8471c25be83a'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:11.396943 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:11.401972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1084/2137] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:21:10 2022] DELETE /identity/v3/users/9bc7ffa8e3cf49fe9dcd8471c25be83a => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:21:11.499145 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06596c6-af12-4bd1-819b-2c158a6275e7 service placement] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:11.500735 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d06596c6-af12-4bd1-819b-2c158a6275e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:11.517390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06596c6-af12-4bd1-819b-2c158a6275e7 service placement] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:11.599922 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd6fe4-7c1a-4b46-b046-53fe4103f959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:11.600664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd6fe4-7c1a-4b46-b046-53fe4103f959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:11.600959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd6fe4-7c1a-4b46-b046-53fe4103f959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:11.601339 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd6fe4-7c1a-4b46-b046-53fe4103f959 admin admin] PATH_INFO: `/v3/users/c13bcda4000d48d6ab9ca7a9bce71fdd` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:11.602628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd6fe4-7c1a-4b46-b046-53fe4103f959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c13bcda4000d48d6ab9ca7a9bce71fdd)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:11.604889 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd6fe4-7c1a-4b46-b046-53fe4103f959 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:11.710439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90efa2-04e3-4c2f-8319-61b9523cf05a admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:11.712043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-1b90efa2-04e3-4c2f-8319-61b9523cf05a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:11.728136 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90efa2-04e3-4c2f-8319-61b9523cf05a admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:11.734886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10b614-ba9d-48b2-8a61-f53ec83b9af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:11.737406 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10b614-ba9d-48b2-8a61-f53ec83b9af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:11.737406 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10b614-ba9d-48b2-8a61-f53ec83b9af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:11.738844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10b614-ba9d-48b2-8a61-f53ec83b9af9 admin admin] PATH_INFO: `/v3/projects/a765c7ccb93545e0a90bc7ffc23a0d46` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:11.740922 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10b614-ba9d-48b2-8a61-f53ec83b9af9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a765c7ccb93545e0a90bc7ffc23a0d46)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:11.743784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10b614-ba9d-48b2-8a61-f53ec83b9af9 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:11.792311 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:11.807068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-3bcd6fe4-7c1a-4b46-b046-53fe4103f959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c13bcda4000d48d6ab9ca7a9bce71fdd'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:11.808276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-3bcd6fe4-7c1a-4b46-b046-53fe4103f959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c13bcda4000d48d6ab9ca7a9bce71fdd'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:11.878959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-3bcd6fe4-7c1a-4b46-b046-53fe4103f959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c13bcda4000d48d6ab9ca7a9bce71fdd'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:11.893331 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-3bcd6fe4-7c1a-4b46-b046-53fe4103f959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c13bcda4000d48d6ab9ca7a9bce71fdd'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:11.898190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-8b10b614-ba9d-48b2-8a61-f53ec83b9af9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a765c7ccb93545e0a90bc7ffc23a0d46', 'request_id': 'req-8b10b614-ba9d-48b2-8a61-f53ec83b9af9'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:11.900338 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-3bcd6fe4-7c1a-4b46-b046-53fe4103f959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c13bcda4000d48d6ab9ca7a9bce71fdd'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:11.905947 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-8b10b614-ba9d-48b2-8a61-f53ec83b9af9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a765c7ccb93545e0a90bc7ffc23a0d46', 'request_id': 'req-8b10b614-ba9d-48b2-8a61-f53ec83b9af9'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:11.911121 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1054/2138] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:21:11 2022] DELETE /identity/v3/users/c13bcda4000d48d6ab9ca7a9bce71fdd => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:11.913178 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccf899-9b4d-47da-bb67-ef54dcb0a204 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:11.913567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccf899-9b4d-47da-bb67-ef54dcb0a204 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:11.913949 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccf899-9b4d-47da-bb67-ef54dcb0a204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:11.917513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-8b10b614-ba9d-48b2-8a61-f53ec83b9af9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a765c7ccb93545e0a90bc7ffc23a0d46', 'request_id': 'req-8b10b614-ba9d-48b2-8a61-f53ec83b9af9'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:11.934212 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-14ccf899-9b4d-47da-bb67-ef54dcb0a204 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:11.935552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-14ccf899-9b4d-47da-bb67-ef54dcb0a204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:11.992687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-8b10b614-ba9d-48b2-8a61-f53ec83b9af9 admin admin] The token cache is being invalidate because project a765c7ccb93545e0a90bc7ffc23a0d46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:21:11.993033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-8b10b614-ba9d-48b2-8a61-f53ec83b9af9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:11.993625 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-8b10b614-ba9d-48b2-8a61-f53ec83b9af9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:11.995350 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1085/2139] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:21:11 2022] DELETE /identity/v3/projects/a765c7ccb93545e0a90bc7ffc23a0d46 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:21:11.997584 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1055/2140] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:21:12.001532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d8a6-f296-4910-94c2-d247fdbe7daf None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:12.001532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d8a6-f296-4910-94c2-d247fdbe7daf None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:12.001753 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d8a6-f296-4910-94c2-d247fdbe7daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:12.007550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccf899-9b4d-47da-bb67-ef54dcb0a204 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:12.009168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-14ccf899-9b4d-47da-bb67-ef54dcb0a204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:12.021647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-fd78d8a6-f296-4910-94c2-d247fdbe7daf None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:12.022752 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-fd78d8a6-f296-4910-94c2-d247fdbe7daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:12.030664 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1086/2141] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:21:12.038897 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f22de-272c-4ffd-b52b-046060a677b6 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:12.039192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f22de-272c-4ffd-b52b-046060a677b6 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:12.039457 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f22de-272c-4ffd-b52b-046060a677b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:12.061099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-fd4f22de-272c-4ffd-b52b-046060a677b6 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:12.062300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-fd4f22de-272c-4ffd-b52b-046060a677b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:12.071274 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1087/2142] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:21:12.075522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccf899-9b4d-47da-bb67-ef54dcb0a204 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:12.075660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f22de-272c-4ffd-b52b-046060a677b6 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:12.078657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65505af-57a1-4cae-91b6-6f289bce2915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:12.079245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65505af-57a1-4cae-91b6-6f289bce2915 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:12.079524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65505af-57a1-4cae-91b6-6f289bce2915 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:12.079795 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65505af-57a1-4cae-91b6-6f289bce2915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:12.081585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a65505af-57a1-4cae-91b6-6f289bce2915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:12.085471 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f22de-272c-4ffd-b52b-046060a677b6 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:12.090189 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e7362-8ea7-4c64-8dc3-5361ffa56def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:12.090776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e7362-8ea7-4c64-8dc3-5361ffa56def admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:12.091054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e7362-8ea7-4c64-8dc3-5361ffa56def admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:12.091323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e7362-8ea7-4c64-8dc3-5361ffa56def admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:12.091961 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e7362-8ea7-4c64-8dc3-5361ffa56def admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:21:12.092261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e7362-8ea7-4c64-8dc3-5361ffa56def admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:12.093567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65505af-57a1-4cae-91b6-6f289bce2915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:12.093876 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e7362-8ea7-4c64-8dc3-5361ffa56def admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:12.095067 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65505af-57a1-4cae-91b6-6f289bce2915 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:12.105422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1056/2143] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:21:12.109292 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1088/2144] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:21:12.110291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65505af-57a1-4cae-91b6-6f289bce2915 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:12.120632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65505af-57a1-4cae-91b6-6f289bce2915 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:12.126973 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1639ec7-9e2f-429b-9856-488de52afe9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:12.127793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1639ec7-9e2f-429b-9856-488de52afe9a admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:12.128172 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1639ec7-9e2f-429b-9856-488de52afe9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:12.128699 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1639ec7-9e2f-429b-9856-488de52afe9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:12.129484 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1639ec7-9e2f-429b-9856-488de52afe9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:12.131428 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1639ec7-9e2f-429b-9856-488de52afe9a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:12.239816 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1057/2145] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:21:12 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:21:12.244124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e7362-8ea7-4c64-8dc3-5361ffa56def admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:12.254125 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e7362-8ea7-4c64-8dc3-5361ffa56def admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:12.379067 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465dfb77-db6c-40df-aa06-87e3bf2e2b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:12.380546 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dfb77-db6c-40df-aa06-87e3bf2e2b1c admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:12.383836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dfb77-db6c-40df-aa06-87e3bf2e2b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:12.383836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dfb77-db6c-40df-aa06-87e3bf2e2b1c admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:12.383836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465dfb77-db6c-40df-aa06-87e3bf2e2b1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:12.384918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465dfb77-db6c-40df-aa06-87e3bf2e2b1c admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:12.494849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1089/2146] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:21:12 2022] POST /identity/v3/users => generated 289 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:21:12.496104 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1639ec7-9e2f-429b-9856-488de52afe9a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:12.507715 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1639ec7-9e2f-429b-9856-488de52afe9a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:12.515766 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760fee0-1fef-453e-817a-74553725f169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:12.516345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760fee0-1fef-453e-817a-74553725f169 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:12.516629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760fee0-1fef-453e-817a-74553725f169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:12.516929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760fee0-1fef-453e-817a-74553725f169 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:12.517590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760fee0-1fef-453e-817a-74553725f169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:12.519966 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760fee0-1fef-453e-817a-74553725f169 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:12.535113 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1058/2147] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:21:12.536715 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465dfb77-db6c-40df-aa06-87e3bf2e2b1c admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:12.546832 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465dfb77-db6c-40df-aa06-87e3bf2e2b1c admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:12.551601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db154bd-1049-45a7-acfd-56fdbac19c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:12.552231 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db154bd-1049-45a7-acfd-56fdbac19c6f admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:12.552516 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db154bd-1049-45a7-acfd-56fdbac19c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:12.552823 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db154bd-1049-45a7-acfd-56fdbac19c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:12.553508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db154bd-1049-45a7-acfd-56fdbac19c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:12.555108 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db154bd-1049-45a7-acfd-56fdbac19c6f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:12.571789 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1090/2148] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:21:12.577998 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760fee0-1fef-453e-817a-74553725f169 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:12.584262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760fee0-1fef-453e-817a-74553725f169 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:12.588872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c3bd5-e516-49be-9221-44d5fafb0c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:12.589524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c3bd5-e516-49be-9221-44d5fafb0c26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:12.589805 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c3bd5-e516-49be-9221-44d5fafb0c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:12.590078 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c3bd5-e516-49be-9221-44d5fafb0c26 admin admin] PATH_INFO: `/v3/projects/c0fbadb872e74c57813900e5ebb6ac88/users/d3f9fac4e1de488b9964f988d238483a/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:12.605907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c3bd5-e516-49be-9221-44d5fafb0c26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0fbadb872e74c57813900e5ebb6ac88, user_id=d3f9fac4e1de488b9964f988d238483a, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:12.608659 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c3bd5-e516-49be-9221-44d5fafb0c26 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:12.698254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1059/2149] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:21:12 2022] PUT /identity/v3/projects/c0fbadb872e74c57813900e5ebb6ac88/users/d3f9fac4e1de488b9964f988d238483a/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:12.711217 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82e74f-1ed1-4982-ae81-71cc23b57641 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:12.711806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82e74f-1ed1-4982-ae81-71cc23b57641 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:12.711806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82e74f-1ed1-4982-ae81-71cc23b57641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:12.743156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-af82e74f-1ed1-4982-ae81-71cc23b57641 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:12.744619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-af82e74f-1ed1-4982-ae81-71cc23b57641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:12.777852 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:12.799714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c3bd5-e516-49be-9221-44d5fafb0c26 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:12.801536 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d72c3bd5-e516-49be-9221-44d5fafb0c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:12.817703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:12.820771 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1091/2150] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:21:12.840677 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82e74f-1ed1-4982-ae81-71cc23b57641 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:12.842416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-af82e74f-1ed1-4982-ae81-71cc23b57641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:12.905657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c3bd5-e516-49be-9221-44d5fafb0c26 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:12.909326 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82e74f-1ed1-4982-ae81-71cc23b57641 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:12.911575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:12.912217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:12.912495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:12.916141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a admin admin] PATH_INFO: `/v3/projects/867a9ed9454d4c6c8bd5b666793bf5c6` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:12.916141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=867a9ed9454d4c6c8bd5b666793bf5c6)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:12.916141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:12.919662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74f5ec-4bc1-4438-b965-36e6084dcad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:12.920284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74f5ec-4bc1-4438-b965-36e6084dcad2 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:12.920564 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74f5ec-4bc1-4438-b965-36e6084dcad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:12.920871 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74f5ec-4bc1-4438-b965-36e6084dcad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:12.935969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74f5ec-4bc1-4438-b965-36e6084dcad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:12.937789 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74f5ec-4bc1-4438-b965-36e6084dcad2 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:12.952030 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1092/2151] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:21:13.068520 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '867a9ed9454d4c6c8bd5b666793bf5c6', 'request_id': 'req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:13.074503 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '867a9ed9454d4c6c8bd5b666793bf5c6', 'request_id': 'req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:13.081065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '867a9ed9454d4c6c8bd5b666793bf5c6', 'request_id': 'req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:13.153775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a admin admin] The token cache is being invalidate because project 867a9ed9454d4c6c8bd5b666793bf5c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:21:13.154069 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:13.154857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:13.157486 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1060/2152] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:21:12 2022] DELETE /identity/v3/projects/867a9ed9454d4c6c8bd5b666793bf5c6 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:13.162376 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74f5ec-4bc1-4438-b965-36e6084dcad2 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:13.172080 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74f5ec-4bc1-4438-b965-36e6084dcad2 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:13.241541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf48b3-aab3-43a7-b758-dabd59770e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:13.242230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf48b3-aab3-43a7-b758-dabd59770e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:13.242517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf48b3-aab3-43a7-b758-dabd59770e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:13.242791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf48b3-aab3-43a7-b758-dabd59770e92 admin admin] PATH_INFO: `/v3/users/a269e68cb5c94ad89b2385e642c25056` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:13.243992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf48b3-aab3-43a7-b758-dabd59770e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a269e68cb5c94ad89b2385e642c25056)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:13.246204 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf48b3-aab3-43a7-b758-dabd59770e92 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:13.536813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-73cf48b3-aab3-43a7-b758-dabd59770e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a269e68cb5c94ad89b2385e642c25056'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:13.611523 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-73cf48b3-aab3-43a7-b758-dabd59770e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a269e68cb5c94ad89b2385e642c25056'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:13.612151 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-73cf48b3-aab3-43a7-b758-dabd59770e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a269e68cb5c94ad89b2385e642c25056'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:13.620010 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-73cf48b3-aab3-43a7-b758-dabd59770e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a269e68cb5c94ad89b2385e642c25056'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:13.633226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:13.634621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-73cf48b3-aab3-43a7-b758-dabd59770e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a269e68cb5c94ad89b2385e642c25056'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:13.641101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1093/2153] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:21:13 2022] DELETE /identity/v3/users/a269e68cb5c94ad89b2385e642c25056 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:14.087305 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:14.089596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:14.110967 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eaaf36-cc4a-4104-bff3-8dfafc61eb6a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:14.180318 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bed61-aa22-4148-ab87-f9759a79485a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:14.180982 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bed61-aa22-4148-ab87-f9759a79485a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:14.181343 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bed61-aa22-4148-ab87-f9759a79485a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:14.181739 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bed61-aa22-4148-ab87-f9759a79485a admin admin] PATH_INFO: `/v3/projects/1f3ea8da62044a55989b651128d063e9` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:14.182858 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bed61-aa22-4148-ab87-f9759a79485a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f3ea8da62044a55989b651128d063e9)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:14.184780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bed61-aa22-4148-ab87-f9759a79485a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:14.301107 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:14.310020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-692bed61-aa22-4148-ab87-f9759a79485a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f3ea8da62044a55989b651128d063e9', 'request_id': 'req-692bed61-aa22-4148-ab87-f9759a79485a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:14.317209 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-692bed61-aa22-4148-ab87-f9759a79485a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f3ea8da62044a55989b651128d063e9', 'request_id': 'req-692bed61-aa22-4148-ab87-f9759a79485a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:14.324920 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-692bed61-aa22-4148-ab87-f9759a79485a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f3ea8da62044a55989b651128d063e9', 'request_id': 'req-692bed61-aa22-4148-ab87-f9759a79485a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:14.442852 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-692bed61-aa22-4148-ab87-f9759a79485a admin admin] The token cache is being invalidate because project 1f3ea8da62044a55989b651128d063e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:21:14.443142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-692bed61-aa22-4148-ab87-f9759a79485a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:14.443756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-692bed61-aa22-4148-ab87-f9759a79485a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:14.445268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1061/2154] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:21:14 2022] DELETE /identity/v3/projects/1f3ea8da62044a55989b651128d063e9 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:14.449259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf48b3-aab3-43a7-b758-dabd59770e92 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:14.450636 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-73cf48b3-aab3-43a7-b758-dabd59770e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:14.464056 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf48b3-aab3-43a7-b758-dabd59770e92 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:14.542692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a56d3a-1493-4fdf-9233-5e45375d6741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:14.543339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a56d3a-1493-4fdf-9233-5e45375d6741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:14.543616 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a56d3a-1493-4fdf-9233-5e45375d6741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:14.543887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a56d3a-1493-4fdf-9233-5e45375d6741 admin admin] PATH_INFO: `/v3/users/4ec908f75f5949cfb124842c077ceded` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:14.545314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a56d3a-1493-4fdf-9233-5e45375d6741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ec908f75f5949cfb124842c077ceded)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:14.547235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a56d3a-1493-4fdf-9233-5e45375d6741 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:14.876881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-56a56d3a-1493-4fdf-9233-5e45375d6741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ec908f75f5949cfb124842c077ceded'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:14.940931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-56a56d3a-1493-4fdf-9233-5e45375d6741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ec908f75f5949cfb124842c077ceded'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:14.941502 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-56a56d3a-1493-4fdf-9233-5e45375d6741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ec908f75f5949cfb124842c077ceded'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:14.948992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-56a56d3a-1493-4fdf-9233-5e45375d6741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ec908f75f5949cfb124842c077ceded'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:14.970769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-56a56d3a-1493-4fdf-9233-5e45375d6741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ec908f75f5949cfb124842c077ceded'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:14.977780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1094/2155] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:21:14 2022] DELETE /identity/v3/users/4ec908f75f5949cfb124842c077ceded => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:15.804287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bed61-aa22-4148-ab87-f9759a79485a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:15.808794 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-692bed61-aa22-4148-ab87-f9759a79485a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:15.825535 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bed61-aa22-4148-ab87-f9759a79485a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:15.967249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440d184-9a86-47d9-98bd-a558504f0aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:15.967854 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440d184-9a86-47d9-98bd-a558504f0aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:15.968189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440d184-9a86-47d9-98bd-a558504f0aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:15.968456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440d184-9a86-47d9-98bd-a558504f0aee admin admin] PATH_INFO: `/v3/projects/e4d0b7698c864f50a4225f411f666ca2` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:15.971248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440d184-9a86-47d9-98bd-a558504f0aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4d0b7698c864f50a4225f411f666ca2)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:15.973851 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440d184-9a86-47d9-98bd-a558504f0aee admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:16.115079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b440d184-9a86-47d9-98bd-a558504f0aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4d0b7698c864f50a4225f411f666ca2', 'request_id': 'req-b440d184-9a86-47d9-98bd-a558504f0aee'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:16.120461 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b440d184-9a86-47d9-98bd-a558504f0aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4d0b7698c864f50a4225f411f666ca2', 'request_id': 'req-b440d184-9a86-47d9-98bd-a558504f0aee'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:16.126807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b440d184-9a86-47d9-98bd-a558504f0aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4d0b7698c864f50a4225f411f666ca2', 'request_id': 'req-b440d184-9a86-47d9-98bd-a558504f0aee'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:16.237823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b440d184-9a86-47d9-98bd-a558504f0aee admin admin] The token cache is being invalidate because project e4d0b7698c864f50a4225f411f666ca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:21:16.238154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b440d184-9a86-47d9-98bd-a558504f0aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:16.238756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b440d184-9a86-47d9-98bd-a558504f0aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:16.240562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1062/2156] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:21:15 2022] DELETE /identity/v3/projects/e4d0b7698c864f50a4225f411f666ca2 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:16.245427 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510891f-ea8b-4558-ad50-0bdd48eb77b2 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:16.246290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510891f-ea8b-4558-ad50-0bdd48eb77b2 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:16.246573 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510891f-ea8b-4558-ad50-0bdd48eb77b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:16.269234 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9510891f-ea8b-4558-ad50-0bdd48eb77b2 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:16.270622 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9510891f-ea8b-4558-ad50-0bdd48eb77b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:16.334112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1095/2157] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:16.342787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a94c1-b7a7-4dcc-8ffb-b70e1ce87512 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:16.343100 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a94c1-b7a7-4dcc-8ffb-b70e1ce87512 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:16.343380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a94c1-b7a7-4dcc-8ffb-b70e1ce87512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:16.368432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-aa0a94c1-b7a7-4dcc-8ffb-b70e1ce87512 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:16.369996 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-aa0a94c1-b7a7-4dcc-8ffb-b70e1ce87512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:16.387263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1063/2158] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:16.387461 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510891f-ea8b-4558-ad50-0bdd48eb77b2 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:16.400828 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510891f-ea8b-4558-ad50-0bdd48eb77b2 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:16.402895 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd983b2d-652f-4c82-9a18-97047c73999f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:16.403528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd983b2d-652f-4c82-9a18-97047c73999f admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:16.403828 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd983b2d-652f-4c82-9a18-97047c73999f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:16.404101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd983b2d-652f-4c82-9a18-97047c73999f admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:16.404784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd983b2d-652f-4c82-9a18-97047c73999f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:21:16.405087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd983b2d-652f-4c82-9a18-97047c73999f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:16.406697 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd983b2d-652f-4c82-9a18-97047c73999f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:16.427366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a94c1-b7a7-4dcc-8ffb-b70e1ce87512 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:16.437943 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1096/2159] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:21:16.442123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a94c1-b7a7-4dcc-8ffb-b70e1ce87512 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:16.449494 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c5d2c-bb8f-4a85-9eaf-4d2a40cc86e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:16.450851 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c5d2c-bb8f-4a85-9eaf-4d2a40cc86e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:16.451508 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c5d2c-bb8f-4a85-9eaf-4d2a40cc86e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:16.452087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c5d2c-bb8f-4a85-9eaf-4d2a40cc86e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:16.452791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c5d2c-bb8f-4a85-9eaf-4d2a40cc86e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:16.456844 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c5d2c-bb8f-4a85-9eaf-4d2a40cc86e2 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:16.614689 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1064/2160] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:21:16 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:21:16.622375 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd983b2d-652f-4c82-9a18-97047c73999f admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:16.629649 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:16.636527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd983b2d-652f-4c82-9a18-97047c73999f admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:16.732037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ca5c5-f161-43e4-85c8-166e35cd1436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:16.732807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ca5c5-f161-43e4-85c8-166e35cd1436 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:16.733161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ca5c5-f161-43e4-85c8-166e35cd1436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:16.733509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ca5c5-f161-43e4-85c8-166e35cd1436 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:16.734212 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ca5c5-f161-43e4-85c8-166e35cd1436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:16.736857 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ca5c5-f161-43e4-85c8-166e35cd1436 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:16.846812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1097/2161] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:21:16 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:21:16.851268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c5d2c-bb8f-4a85-9eaf-4d2a40cc86e2 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:16.864672 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c5d2c-bb8f-4a85-9eaf-4d2a40cc86e2 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:16.872692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c08e2-1cad-4152-b1d1-dd551f1e3cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:16.873608 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c08e2-1cad-4152-b1d1-dd551f1e3cbe admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:16.874824 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c08e2-1cad-4152-b1d1-dd551f1e3cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:16.880342 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c08e2-1cad-4152-b1d1-dd551f1e3cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:16.880342 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c08e2-1cad-4152-b1d1-dd551f1e3cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:16.880342 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c08e2-1cad-4152-b1d1-dd551f1e3cbe admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:16.891150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1065/2162] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:21:16.896177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ca5c5-f161-43e4-85c8-166e35cd1436 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:16.906361 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ca5c5-f161-43e4-85c8-166e35cd1436 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:16.912370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e686d-9502-4128-8f5d-7f21560b0346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:16.913219 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e686d-9502-4128-8f5d-7f21560b0346 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:16.913657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e686d-9502-4128-8f5d-7f21560b0346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:16.914087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e686d-9502-4128-8f5d-7f21560b0346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:16.914783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e686d-9502-4128-8f5d-7f21560b0346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:16.916487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e686d-9502-4128-8f5d-7f21560b0346 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:16.950628 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1098/2163] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:21:16.952099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c08e2-1cad-4152-b1d1-dd551f1e3cbe admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:16.963322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c08e2-1cad-4152-b1d1-dd551f1e3cbe admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:16.975483 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522627a-3118-4a92-a847-d3efc035bbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:16.976472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522627a-3118-4a92-a847-d3efc035bbc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:16.976825 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522627a-3118-4a92-a847-d3efc035bbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:16.977181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522627a-3118-4a92-a847-d3efc035bbc1 admin admin] PATH_INFO: `/v3/projects/f1b3e4986e72478bb46a86e1e9e3355c/users/07db808b777b4407ac05f3f0fa2394f8/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:16.995246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:16.999334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522627a-3118-4a92-a847-d3efc035bbc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b3e4986e72478bb46a86e1e9e3355c, user_id=07db808b777b4407ac05f3f0fa2394f8, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:17.006031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522627a-3118-4a92-a847-d3efc035bbc1 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:17.123112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1066/2164] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:21:16 2022] PUT /identity/v3/projects/f1b3e4986e72478bb46a86e1e9e3355c/users/07db808b777b4407ac05f3f0fa2394f8/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:17.130098 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8612dd-8088-4b3b-b475-fb9bdd33079c None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:17.130836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8612dd-8088-4b3b-b475-fb9bdd33079c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:17.131702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8612dd-8088-4b3b-b475-fb9bdd33079c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:17.152009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-6e8612dd-8088-4b3b-b475-fb9bdd33079c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:17.153420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-6e8612dd-8088-4b3b-b475-fb9bdd33079c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:17.315775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1099/2165] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:17.321517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522627a-3118-4a92-a847-d3efc035bbc1 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:17.323422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2522627a-3118-4a92-a847-d3efc035bbc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:17.440472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522627a-3118-4a92-a847-d3efc035bbc1 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:17.448378 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439415fb-1dd9-4711-8744-a336c22d12b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:17.449143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439415fb-1dd9-4711-8744-a336c22d12b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:17.449483 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439415fb-1dd9-4711-8744-a336c22d12b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:17.449763 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439415fb-1dd9-4711-8744-a336c22d12b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:17.474265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439415fb-1dd9-4711-8744-a336c22d12b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:17.476225 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439415fb-1dd9-4711-8744-a336c22d12b0 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:17.491170 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1067/2166] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:21:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:21:19.919466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53fb6c-739a-4e01-995b-a757c07a42a7 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:19.919929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53fb6c-739a-4e01-995b-a757c07a42a7 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:19.920160 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53fb6c-739a-4e01-995b-a757c07a42a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:20.030207 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9c53fb6c-739a-4e01-995b-a757c07a42a7 None None] MFA Rules not processed for user `07db808b777b4407ac05f3f0fa2394f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:20.031499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9c53fb6c-739a-4e01-995b-a757c07a42a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:20.238486 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799773-277d-4d77-814f-180bd4acdc94 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:20.239015 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799773-277d-4d77-814f-180bd4acdc94 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:20.239015 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799773-277d-4d77-814f-180bd4acdc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:20.239284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1100/2167] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:20.252500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53fb6c-739a-4e01-995b-a757c07a42a7 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:20.255176 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9c53fb6c-739a-4e01-995b-a757c07a42a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:20.289944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:20.327782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-15799773-277d-4d77-814f-180bd4acdc94 None None] MFA Rules not processed for user `d3f9fac4e1de488b9964f988d238483a`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:20.330866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-15799773-277d-4d77-814f-180bd4acdc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:20.364623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53fb6c-739a-4e01-995b-a757c07a42a7 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:20.370271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bca152-039b-4899-b027-5246235161b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:20.371252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bca152-039b-4899-b027-5246235161b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:20.371706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bca152-039b-4899-b027-5246235161b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:20.372087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bca152-039b-4899-b027-5246235161b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:20.383057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bca152-039b-4899-b027-5246235161b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:20.384507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bca152-039b-4899-b027-5246235161b9 service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:20.395352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1101/2168] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:21:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:21:20.418083 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1068/2169] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:21:20.435933 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bca152-039b-4899-b027-5246235161b9 service cinder] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:20.435933 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-37bca152-039b-4899-b027-5246235161b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:20.544940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bca152-039b-4899-b027-5246235161b9 service cinder] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:20.553563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e64efb-6fdf-4a3a-ba65-821aae7739ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:20.554192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e64efb-6fdf-4a3a-ba65-821aae7739ca service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:20.554952 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e64efb-6fdf-4a3a-ba65-821aae7739ca service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:20.555241 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e64efb-6fdf-4a3a-ba65-821aae7739ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:20.565408 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e64efb-6fdf-4a3a-ba65-821aae7739ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:20.567719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e64efb-6fdf-4a3a-ba65-821aae7739ca service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:20.579424 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1102/2170] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:21:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:21:23.338395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6670e-306d-4b0f-a659-9e5992c98401 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:23.339357 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6670e-306d-4b0f-a659-9e5992c98401 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:23.339848 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6670e-306d-4b0f-a659-9e5992c98401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:23.381029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-7dd6670e-306d-4b0f-a659-9e5992c98401 None None] MFA Rules not processed for user `07db808b777b4407ac05f3f0fa2394f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:23.382418 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-7dd6670e-306d-4b0f-a659-9e5992c98401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:23.413113 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1069/2171] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:24.062766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c9aa7-89a7-421c-96ff-9dbf9045ea03 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:24.063238 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c9aa7-89a7-421c-96ff-9dbf9045ea03 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:24.063338 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c9aa7-89a7-421c-96ff-9dbf9045ea03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:24.085533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-029c9aa7-89a7-421c-96ff-9dbf9045ea03 None None] MFA Rules not processed for user `d3f9fac4e1de488b9964f988d238483a`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:24.086783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-029c9aa7-89a7-421c-96ff-9dbf9045ea03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:24.094639 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1103/2172] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:26.574851 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587e715-b6f3-416e-9b80-6eb1ebcd26ae None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:26.575446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587e715-b6f3-416e-9b80-6eb1ebcd26ae None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:26.575446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587e715-b6f3-416e-9b80-6eb1ebcd26ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:26.618825 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-8587e715-b6f3-416e-9b80-6eb1ebcd26ae None None] MFA Rules not processed for user `011f5d15687d4a55ac595c8f067d6f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:26.621501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8587e715-b6f3-416e-9b80-6eb1ebcd26ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:26.753141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1070/2173] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:34.077787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=58 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:36.738113 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:46.099775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c9aa7-89a7-421c-96ff-9dbf9045ea03 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:46.101390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-029c9aa7-89a7-421c-96ff-9dbf9045ea03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:46.115105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c9aa7-89a7-421c-96ff-9dbf9045ea03 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:46.119606 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73371d-e02c-419a-a005-2ffdf2d641b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:46.120237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73371d-e02c-419a-a005-2ffdf2d641b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:46.120590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73371d-e02c-419a-a005-2ffdf2d641b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:46.120825 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73371d-e02c-419a-a005-2ffdf2d641b7 admin admin] PATH_INFO: `/v3/users/dd4fa310d8594198a47ee4f374045fe4` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:46.121958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73371d-e02c-419a-a005-2ffdf2d641b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd4fa310d8594198a47ee4f374045fe4)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:46.123877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73371d-e02c-419a-a005-2ffdf2d641b7 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:46.297040 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-6a73371d-e02c-419a-a005-2ffdf2d641b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd4fa310d8594198a47ee4f374045fe4'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:46.378086 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-6a73371d-e02c-419a-a005-2ffdf2d641b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd4fa310d8594198a47ee4f374045fe4'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:46.378647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-6a73371d-e02c-419a-a005-2ffdf2d641b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd4fa310d8594198a47ee4f374045fe4'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:46.385819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-6a73371d-e02c-419a-a005-2ffdf2d641b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd4fa310d8594198a47ee4f374045fe4'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:46.398236 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-6a73371d-e02c-419a-a005-2ffdf2d641b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd4fa310d8594198a47ee4f374045fe4'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:46.404073 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1104/2174] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:21:46 2022] DELETE /identity/v3/users/dd4fa310d8594198a47ee4f374045fe4 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:46.409610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ae583-6651-4b73-a889-ea3d7bd74f37 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:46.409610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ae583-6651-4b73-a889-ea3d7bd74f37 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:46.409610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ae583-6651-4b73-a889-ea3d7bd74f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:46.430251 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-0e7ae583-6651-4b73-a889-ea3d7bd74f37 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:46.431189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0e7ae583-6651-4b73-a889-ea3d7bd74f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:46.506687 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1071/2175] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:46.518022 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73371d-e02c-419a-a005-2ffdf2d641b7 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:46.519778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-6a73371d-e02c-419a-a005-2ffdf2d641b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:46.582209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73371d-e02c-419a-a005-2ffdf2d641b7 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:46.586000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0439a31-55d6-48fb-ad01-a84ec5f3fa4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:46.586612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0439a31-55d6-48fb-ad01-a84ec5f3fa4b service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:46.586882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0439a31-55d6-48fb-ad01-a84ec5f3fa4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:46.587155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0439a31-55d6-48fb-ad01-a84ec5f3fa4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:46.597231 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0439a31-55d6-48fb-ad01-a84ec5f3fa4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:46.598760 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0439a31-55d6-48fb-ad01-a84ec5f3fa4b service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:46.608956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1105/2176] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:21:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:21:47.130218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ae583-6651-4b73-a889-ea3d7bd74f37 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:47.131709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0e7ae583-6651-4b73-a889-ea3d7bd74f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:47.145447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ae583-6651-4b73-a889-ea3d7bd74f37 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:47.150096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a2a7a-e55c-4564-89d8-a6151a611c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:47.150745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a2a7a-e55c-4564-89d8-a6151a611c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:47.151004 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a2a7a-e55c-4564-89d8-a6151a611c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:47.151275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a2a7a-e55c-4564-89d8-a6151a611c1d admin admin] PATH_INFO: `/v3/projects/b338e860be6b4c1ea00ad1ab95b0317e` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:47.153554 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a2a7a-e55c-4564-89d8-a6151a611c1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b338e860be6b4c1ea00ad1ab95b0317e)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:47.155442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a2a7a-e55c-4564-89d8-a6151a611c1d admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:47.272137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-558a2a7a-e55c-4564-89d8-a6151a611c1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b338e860be6b4c1ea00ad1ab95b0317e', 'request_id': 'req-558a2a7a-e55c-4564-89d8-a6151a611c1d'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:47.277954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-558a2a7a-e55c-4564-89d8-a6151a611c1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b338e860be6b4c1ea00ad1ab95b0317e', 'request_id': 'req-558a2a7a-e55c-4564-89d8-a6151a611c1d'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:47.285301 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-558a2a7a-e55c-4564-89d8-a6151a611c1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b338e860be6b4c1ea00ad1ab95b0317e', 'request_id': 'req-558a2a7a-e55c-4564-89d8-a6151a611c1d'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:47.349823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-558a2a7a-e55c-4564-89d8-a6151a611c1d admin admin] The token cache is being invalidate because project b338e860be6b4c1ea00ad1ab95b0317e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:21:47.350159 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-558a2a7a-e55c-4564-89d8-a6151a611c1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:47.350831 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-558a2a7a-e55c-4564-89d8-a6151a611c1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:21:47.352322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1072/2177] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:21:47 2022] DELETE /identity/v3/projects/b338e860be6b4c1ea00ad1ab95b0317e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:47.357240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6f0ea-d615-47a6-b8de-6ba4fe23606f None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:47.357736 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6f0ea-d615-47a6-b8de-6ba4fe23606f None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:47.357851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6f0ea-d615-47a6-b8de-6ba4fe23606f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:47.377929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-ebe6f0ea-d615-47a6-b8de-6ba4fe23606f None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:47.379256 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ebe6f0ea-d615-47a6-b8de-6ba4fe23606f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:47.437409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1106/2178] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:47.445942 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f564897-30ab-4ad0-84f1-0d8be57d468c None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:47.446433 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f564897-30ab-4ad0-84f1-0d8be57d468c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:47.446595 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f564897-30ab-4ad0-84f1-0d8be57d468c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:47.466955 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-2f564897-30ab-4ad0-84f1-0d8be57d468c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:47.468153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2f564897-30ab-4ad0-84f1-0d8be57d468c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:47.476274 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1073/2179] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:47.480552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6f0ea-d615-47a6-b8de-6ba4fe23606f None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:47.491204 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6f0ea-d615-47a6-b8de-6ba4fe23606f None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:47.496012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218ab04-8bd3-4d9b-9a89-2cc68f274767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:47.496607 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218ab04-8bd3-4d9b-9a89-2cc68f274767 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:47.498023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218ab04-8bd3-4d9b-9a89-2cc68f274767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:47.498023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218ab04-8bd3-4d9b-9a89-2cc68f274767 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:47.498023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218ab04-8bd3-4d9b-9a89-2cc68f274767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:21:47.498284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218ab04-8bd3-4d9b-9a89-2cc68f274767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:47.499778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218ab04-8bd3-4d9b-9a89-2cc68f274767 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:47.511740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1107/2180] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:21:47.516247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f564897-30ab-4ad0-84f1-0d8be57d468c None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:47.525805 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f564897-30ab-4ad0-84f1-0d8be57d468c None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:47.530694 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514ba4b-9744-4f44-a5c2-5a624019fd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:47.531319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514ba4b-9744-4f44-a5c2-5a624019fd9a admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:47.531601 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514ba4b-9744-4f44-a5c2-5a624019fd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:47.532433 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514ba4b-9744-4f44-a5c2-5a624019fd9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:47.532531 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514ba4b-9744-4f44-a5c2-5a624019fd9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:47.534789 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514ba4b-9744-4f44-a5c2-5a624019fd9a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:47.615236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1074/2181] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:21:47 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:21:47.619403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218ab04-8bd3-4d9b-9a89-2cc68f274767 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:47.624789 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:47.630883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218ab04-8bd3-4d9b-9a89-2cc68f274767 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:47.687568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec3fb7-7163-458d-bd99-bf8d8654bbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:47.688142 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec3fb7-7163-458d-bd99-bf8d8654bbde admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:47.688440 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec3fb7-7163-458d-bd99-bf8d8654bbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:47.689313 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec3fb7-7163-458d-bd99-bf8d8654bbde admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:47.690063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec3fb7-7163-458d-bd99-bf8d8654bbde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:47.691975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec3fb7-7163-458d-bd99-bf8d8654bbde admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:47.825427 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1108/2182] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:21:47 2022] POST /identity/v3/users => generated 286 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:21:47.826668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514ba4b-9744-4f44-a5c2-5a624019fd9a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:47.845349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514ba4b-9744-4f44-a5c2-5a624019fd9a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:47.845349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d4136-5239-4757-9a9f-2e8e5ad1b87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:47.845349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d4136-5239-4757-9a9f-2e8e5ad1b87b admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:47.845831 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d4136-5239-4757-9a9f-2e8e5ad1b87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:47.846342 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d4136-5239-4757-9a9f-2e8e5ad1b87b admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:47.847686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d4136-5239-4757-9a9f-2e8e5ad1b87b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:47.849532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d4136-5239-4757-9a9f-2e8e5ad1b87b admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:47.866619 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1075/2183] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:21:47.867780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec3fb7-7163-458d-bd99-bf8d8654bbde admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:47.879148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec3fb7-7163-458d-bd99-bf8d8654bbde admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:47.888099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f43158-1b7a-44cd-8ae3-10a00c779a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:47.889546 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f43158-1b7a-44cd-8ae3-10a00c779a12 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:47.890103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f43158-1b7a-44cd-8ae3-10a00c779a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:47.890386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f43158-1b7a-44cd-8ae3-10a00c779a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:47.891135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f43158-1b7a-44cd-8ae3-10a00c779a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:47.893121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f43158-1b7a-44cd-8ae3-10a00c779a12 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:47.909154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1109/2184] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:21:47.911840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d4136-5239-4757-9a9f-2e8e5ad1b87b admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:47.927617 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d4136-5239-4757-9a9f-2e8e5ad1b87b admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:47.933220 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2108a-3f8e-440e-b8fa-23fba1bbfdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:47.933929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2108a-3f8e-440e-b8fa-23fba1bbfdd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:47.934278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2108a-3f8e-440e-b8fa-23fba1bbfdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:47.934726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2108a-3f8e-440e-b8fa-23fba1bbfdd6 admin admin] PATH_INFO: `/v3/projects/58a76d86371c448a8ff0596da2235ba5/users/013b2068b6e8422781a93b4ecef436bc/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:47.952361 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2108a-3f8e-440e-b8fa-23fba1bbfdd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a76d86371c448a8ff0596da2235ba5, user_id=013b2068b6e8422781a93b4ecef436bc, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:47.955795 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2108a-3f8e-440e-b8fa-23fba1bbfdd6 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:48.077881 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1076/2185] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:21:47 2022] PUT /identity/v3/projects/58a76d86371c448a8ff0596da2235ba5/users/013b2068b6e8422781a93b4ecef436bc/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:21:48.089286 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5816c9-a7ea-46cc-8e5f-f9920a747cdb None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:48.089577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5816c9-a7ea-46cc-8e5f-f9920a747cdb None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:48.089855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5816c9-a7ea-46cc-8e5f-f9920a747cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:48.129998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-5a5816c9-a7ea-46cc-8e5f-f9920a747cdb None None] MFA Rules not processed for user `013b2068b6e8422781a93b4ecef436bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:48.131282 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-5a5816c9-a7ea-46cc-8e5f-f9920a747cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:48.206969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1110/2186] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:21:48.217552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2108a-3f8e-440e-b8fa-23fba1bbfdd6 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:21:48.219270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-27a2108a-3f8e-440e-b8fa-23fba1bbfdd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:48.243109 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:21:48.278429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2108a-3f8e-440e-b8fa-23fba1bbfdd6 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:21:48.282799 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97520d3b-9767-417a-82a2-d41a8cdab4eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:21:48.283960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97520d3b-9767-417a-82a2-d41a8cdab4eb service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:48.284252 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97520d3b-9767-417a-82a2-d41a8cdab4eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:48.284531 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97520d3b-9767-417a-82a2-d41a8cdab4eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:48.293369 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97520d3b-9767-417a-82a2-d41a8cdab4eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:21:48.295113 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97520d3b-9767-417a-82a2-d41a8cdab4eb service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:21:48.303575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1077/2187] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:21:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:21:51.105912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40863915-7643-4887-9285-5c460f9f6b89 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:21:51.106449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40863915-7643-4887-9285-5c460f9f6b89 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:21:51.106610 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40863915-7643-4887-9285-5c460f9f6b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:21:51.125969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-40863915-7643-4887-9285-5c460f9f6b89 None None] MFA Rules not processed for user `013b2068b6e8422781a93b4ecef436bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:21:51.127298 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-40863915-7643-4887-9285-5c460f9f6b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:21:51.134196 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1111/2188] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:21:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:21:58.343605 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=17 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:22:01.120165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=68,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:22:05.522438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97520d3b-9767-417a-82a2-d41a8cdab4eb service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:05.524818 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-97520d3b-9767-417a-82a2-d41a8cdab4eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:05.628716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97520d3b-9767-417a-82a2-d41a8cdab4eb service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:05.634668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14120483-d06a-432f-bb71-e78f03cc62ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ee8ba21bc184feeaafacab3e69e87bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:05.635390 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14120483-d06a-432f-bb71-e78f03cc62ae service placement] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:05.635770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14120483-d06a-432f-bb71-e78f03cc62ae service placement] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:05.636246 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14120483-d06a-432f-bb71-e78f03cc62ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:05.638262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-14120483-d06a-432f-bb71-e78f03cc62ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:05.650730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14120483-d06a-432f-bb71-e78f03cc62ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:05.658235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14120483-d06a-432f-bb71-e78f03cc62ae service placement] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:05.663682 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1078/2189] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:22:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:22:12.258069 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40863915-7643-4887-9285-5c460f9f6b89 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:12.259562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-40863915-7643-4887-9285-5c460f9f6b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:12.324222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40863915-7643-4887-9285-5c460f9f6b89 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:12.328900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d0921-fdff-49e2-bb01-14dbe241e457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:12.329534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d0921-fdff-49e2-bb01-14dbe241e457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:12.330512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d0921-fdff-49e2-bb01-14dbe241e457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:12.330789 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d0921-fdff-49e2-bb01-14dbe241e457 admin admin] PATH_INFO: `/v3/users/b220740b1c204c968864226ae00a2272` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:12.331920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d0921-fdff-49e2-bb01-14dbe241e457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b220740b1c204c968864226ae00a2272)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:12.333861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d0921-fdff-49e2-bb01-14dbe241e457 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:12.489888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d71d0921-fdff-49e2-bb01-14dbe241e457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b220740b1c204c968864226ae00a2272'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:12.571977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d71d0921-fdff-49e2-bb01-14dbe241e457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b220740b1c204c968864226ae00a2272'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:12.572621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d71d0921-fdff-49e2-bb01-14dbe241e457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b220740b1c204c968864226ae00a2272'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:12.580784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d71d0921-fdff-49e2-bb01-14dbe241e457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b220740b1c204c968864226ae00a2272'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:12.596531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d71d0921-fdff-49e2-bb01-14dbe241e457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b220740b1c204c968864226ae00a2272'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:12.602658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1112/2190] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:22:12 2022] DELETE /identity/v3/users/b220740b1c204c968864226ae00a2272 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:22:12.607261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac352c-8150-4dc3-b74b-1307b7c310d1 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:12.607764 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac352c-8150-4dc3-b74b-1307b7c310d1 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:12.607875 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac352c-8150-4dc3-b74b-1307b7c310d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:12.629572 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-79ac352c-8150-4dc3-b74b-1307b7c310d1 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:12.630777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-79ac352c-8150-4dc3-b74b-1307b7c310d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:12.695894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1079/2191] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:12.711627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d0921-fdff-49e2-bb01-14dbe241e457 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:12.713110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-d71d0921-fdff-49e2-bb01-14dbe241e457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:12.806428 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d0921-fdff-49e2-bb01-14dbe241e457 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:12.810966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a794be-39d8-4892-862e-2936dcd4b6ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:12.812057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a794be-39d8-4892-862e-2936dcd4b6ab service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:12.812057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a794be-39d8-4892-862e-2936dcd4b6ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:12.812239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a794be-39d8-4892-862e-2936dcd4b6ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:12.822778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a794be-39d8-4892-862e-2936dcd4b6ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:12.824230 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a794be-39d8-4892-862e-2936dcd4b6ab service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:12.847162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1113/2192] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:22:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:22:13.744565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac352c-8150-4dc3-b74b-1307b7c310d1 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:13.746025 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-79ac352c-8150-4dc3-b74b-1307b7c310d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:13.758190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac352c-8150-4dc3-b74b-1307b7c310d1 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:13.762797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb20821-86f4-4b06-82aa-38b29eb5ff3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:13.766181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb20821-86f4-4b06-82aa-38b29eb5ff3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:13.766181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb20821-86f4-4b06-82aa-38b29eb5ff3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:13.766181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb20821-86f4-4b06-82aa-38b29eb5ff3c admin admin] PATH_INFO: `/v3/projects/61d4fcdc0ddb4a30b0365284ca0e180c` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:13.766181 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb20821-86f4-4b06-82aa-38b29eb5ff3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=61d4fcdc0ddb4a30b0365284ca0e180c)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:13.767598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb20821-86f4-4b06-82aa-38b29eb5ff3c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:13.923244 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-cbb20821-86f4-4b06-82aa-38b29eb5ff3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61d4fcdc0ddb4a30b0365284ca0e180c', 'request_id': 'req-cbb20821-86f4-4b06-82aa-38b29eb5ff3c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:13.934933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-cbb20821-86f4-4b06-82aa-38b29eb5ff3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61d4fcdc0ddb4a30b0365284ca0e180c', 'request_id': 'req-cbb20821-86f4-4b06-82aa-38b29eb5ff3c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:13.943651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-cbb20821-86f4-4b06-82aa-38b29eb5ff3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61d4fcdc0ddb4a30b0365284ca0e180c', 'request_id': 'req-cbb20821-86f4-4b06-82aa-38b29eb5ff3c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:14.047532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-cbb20821-86f4-4b06-82aa-38b29eb5ff3c admin admin] The token cache is being invalidate because project 61d4fcdc0ddb4a30b0365284ca0e180c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:22:14.048040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-cbb20821-86f4-4b06-82aa-38b29eb5ff3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:14.049458 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-cbb20821-86f4-4b06-82aa-38b29eb5ff3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:14.055013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1080/2193] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:22:13 2022] DELETE /identity/v3/projects/61d4fcdc0ddb4a30b0365284ca0e180c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:22:14.057634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ed023-c57b-49fc-82b7-7e42ae31602c None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:14.058634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ed023-c57b-49fc-82b7-7e42ae31602c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:14.058981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ed023-c57b-49fc-82b7-7e42ae31602c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:14.085050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-2b1ed023-c57b-49fc-82b7-7e42ae31602c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:14.086746 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2b1ed023-c57b-49fc-82b7-7e42ae31602c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:14.114316 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:22:14.181342 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1114/2194] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:14.190846 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4845046-d639-45d4-96cb-618302fee7c1 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:14.191136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4845046-d639-45d4-96cb-618302fee7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:14.191414 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4845046-d639-45d4-96cb-618302fee7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:14.213670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-d4845046-d639-45d4-96cb-618302fee7c1 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:14.215223 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d4845046-d639-45d4-96cb-618302fee7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:14.225312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1081/2195] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:22:14.229918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ed023-c57b-49fc-82b7-7e42ae31602c None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:14.252528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ed023-c57b-49fc-82b7-7e42ae31602c None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:14.259371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4e936-4795-40b7-9051-bab68dd1899f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:14.259993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4e936-4795-40b7-9051-bab68dd1899f admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:14.260287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4e936-4795-40b7-9051-bab68dd1899f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:14.260565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4e936-4795-40b7-9051-bab68dd1899f admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:14.261293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4e936-4795-40b7-9051-bab68dd1899f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:22:14.261609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4e936-4795-40b7-9051-bab68dd1899f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:14.263252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4e936-4795-40b7-9051-bab68dd1899f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:14.276680 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1115/2196] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:22:14.280419 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4845046-d639-45d4-96cb-618302fee7c1 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:14.292790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4845046-d639-45d4-96cb-618302fee7c1 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:14.299252 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d7c53-688b-46de-8de6-03dc4fc171b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:14.299898 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d7c53-688b-46de-8de6-03dc4fc171b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:14.300273 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d7c53-688b-46de-8de6-03dc4fc171b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:14.300692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d7c53-688b-46de-8de6-03dc4fc171b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:14.301415 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d7c53-688b-46de-8de6-03dc4fc171b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:14.303546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d7c53-688b-46de-8de6-03dc4fc171b7 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:14.308429 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:22:14.425939 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1082/2197] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:22:14 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:22:14.427087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4e936-4795-40b7-9051-bab68dd1899f admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:14.438028 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4e936-4795-40b7-9051-bab68dd1899f admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:14.515924 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3ceb0-972a-4a59-bcfa-bd55066a488a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:14.518391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3ceb0-972a-4a59-bcfa-bd55066a488a admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:14.518391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3ceb0-972a-4a59-bcfa-bd55066a488a admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:14.518391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3ceb0-972a-4a59-bcfa-bd55066a488a admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:14.519521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3ceb0-972a-4a59-bcfa-bd55066a488a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:14.521140 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3ceb0-972a-4a59-bcfa-bd55066a488a admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:14.711459 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1116/2198] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:22:14 2022] POST /identity/v3/users => generated 286 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:22:14.719341 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d7c53-688b-46de-8de6-03dc4fc171b7 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:14.730873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d7c53-688b-46de-8de6-03dc4fc171b7 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:14.735421 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabd5d1-0ebc-4d35-89e4-bce9de82d7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:14.736018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabd5d1-0ebc-4d35-89e4-bce9de82d7ad admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:14.736337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabd5d1-0ebc-4d35-89e4-bce9de82d7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:14.736632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabd5d1-0ebc-4d35-89e4-bce9de82d7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:14.737373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabd5d1-0ebc-4d35-89e4-bce9de82d7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:14.738971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabd5d1-0ebc-4d35-89e4-bce9de82d7ad admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:14.751514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1083/2199] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:22:14.756431 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3ceb0-972a-4a59-bcfa-bd55066a488a admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:14.766461 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3ceb0-972a-4a59-bcfa-bd55066a488a admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:14.771928 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99692a-dd84-48a1-a67d-8d2c1f8be85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:14.772555 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99692a-dd84-48a1-a67d-8d2c1f8be85f admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:14.773444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99692a-dd84-48a1-a67d-8d2c1f8be85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:14.773444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99692a-dd84-48a1-a67d-8d2c1f8be85f admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:14.773769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99692a-dd84-48a1-a67d-8d2c1f8be85f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:14.775386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99692a-dd84-48a1-a67d-8d2c1f8be85f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:14.789325 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1117/2200] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:22:14.794023 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabd5d1-0ebc-4d35-89e4-bce9de82d7ad admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:14.805653 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabd5d1-0ebc-4d35-89e4-bce9de82d7ad admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:14.810292 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5e284-8cb2-4647-af4c-7bda71d7130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:14.810997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5e284-8cb2-4647-af4c-7bda71d7130c admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:14.811281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5e284-8cb2-4647-af4c-7bda71d7130c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:14.811559 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5e284-8cb2-4647-af4c-7bda71d7130c admin admin] PATH_INFO: `/v3/projects/e8c500936623419bb21a24d7211d9126/users/c1d2f131e94c43f89524ebaf5a1e24c3/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:14.827301 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5e284-8cb2-4647-af4c-7bda71d7130c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8c500936623419bb21a24d7211d9126, user_id=c1d2f131e94c43f89524ebaf5a1e24c3, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:14.830053 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5e284-8cb2-4647-af4c-7bda71d7130c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:14.948238 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1084/2201] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:22:14 2022] PUT /identity/v3/projects/e8c500936623419bb21a24d7211d9126/users/c1d2f131e94c43f89524ebaf5a1e24c3/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:22:14.959187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4961a-8edd-49f9-9221-a1f1fadcdafe None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:14.959715 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4961a-8edd-49f9-9221-a1f1fadcdafe None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:14.960003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4961a-8edd-49f9-9221-a1f1fadcdafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:14.992959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-07a4961a-8edd-49f9-9221-a1f1fadcdafe None None] MFA Rules not processed for user `c1d2f131e94c43f89524ebaf5a1e24c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:14.994259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-07a4961a-8edd-49f9-9221-a1f1fadcdafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:15.087550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1118/2202] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:22:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:22:15.096082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5e284-8cb2-4647-af4c-7bda71d7130c admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:15.098589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c3d5e284-8cb2-4647-af4c-7bda71d7130c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:15.182098 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5e284-8cb2-4647-af4c-7bda71d7130c admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:15.189005 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3508b9-aabb-466e-ac0e-8d146e88c968 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:15.190005 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3508b9-aabb-466e-ac0e-8d146e88c968 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:15.190485 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3508b9-aabb-466e-ac0e-8d146e88c968 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:15.190948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3508b9-aabb-466e-ac0e-8d146e88c968 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:15.203992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3508b9-aabb-466e-ac0e-8d146e88c968 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:15.207758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3508b9-aabb-466e-ac0e-8d146e88c968 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:15.218329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1085/2203] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:22:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:22:15.257516 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4961a-8edd-49f9-9221-a1f1fadcdafe None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:15.292371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:22:15.343272 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4961a-8edd-49f9-9221-a1f1fadcdafe None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:15.348958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40d86d-6667-48bf-b941-8919a8abf870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:15.349601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40d86d-6667-48bf-b941-8919a8abf870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:15.349889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40d86d-6667-48bf-b941-8919a8abf870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:15.350169 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40d86d-6667-48bf-b941-8919a8abf870 admin admin] PATH_INFO: `/v3/users/c1d2f131e94c43f89524ebaf5a1e24c3` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:15.351403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40d86d-6667-48bf-b941-8919a8abf870 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1d2f131e94c43f89524ebaf5a1e24c3)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:15.353517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40d86d-6667-48bf-b941-8919a8abf870 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:15.609052 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1b40d86d-6667-48bf-b941-8919a8abf870 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1d2f131e94c43f89524ebaf5a1e24c3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:15.677786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1b40d86d-6667-48bf-b941-8919a8abf870 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1d2f131e94c43f89524ebaf5a1e24c3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:15.679289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1b40d86d-6667-48bf-b941-8919a8abf870 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1d2f131e94c43f89524ebaf5a1e24c3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:15.687572 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1b40d86d-6667-48bf-b941-8919a8abf870 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1d2f131e94c43f89524ebaf5a1e24c3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:15.701120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1b40d86d-6667-48bf-b941-8919a8abf870 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1d2f131e94c43f89524ebaf5a1e24c3'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:15.708130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1119/2204] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:22:15 2022] DELETE /identity/v3/users/c1d2f131e94c43f89524ebaf5a1e24c3 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:22:15.712855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894bd61-00bd-4500-a9d0-7be60f9209ef None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:15.713142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894bd61-00bd-4500-a9d0-7be60f9209ef None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:15.713414 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894bd61-00bd-4500-a9d0-7be60f9209ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:15.739465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-2894bd61-00bd-4500-a9d0-7be60f9209ef None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:15.741136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2894bd61-00bd-4500-a9d0-7be60f9209ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:15.817153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1086/2205] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:15.828009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40d86d-6667-48bf-b941-8919a8abf870 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:15.830537 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-1b40d86d-6667-48bf-b941-8919a8abf870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:15.893954 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40d86d-6667-48bf-b941-8919a8abf870 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:15.897558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8379c1-0c15-49bd-a970-49302d3cefe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:15.898161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8379c1-0c15-49bd-a970-49302d3cefe5 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:15.898440 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8379c1-0c15-49bd-a970-49302d3cefe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:15.898712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8379c1-0c15-49bd-a970-49302d3cefe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:15.908621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8379c1-0c15-49bd-a970-49302d3cefe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:15.910076 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8379c1-0c15-49bd-a970-49302d3cefe5 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:15.920754 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1120/2206] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:22:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:22:16.338040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894bd61-00bd-4500-a9d0-7be60f9209ef None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:16.339888 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2894bd61-00bd-4500-a9d0-7be60f9209ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:16.353995 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894bd61-00bd-4500-a9d0-7be60f9209ef None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:16.359123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513be42-5b11-4bff-8c5a-ec226a42b307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:16.359760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513be42-5b11-4bff-8c5a-ec226a42b307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:16.360667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513be42-5b11-4bff-8c5a-ec226a42b307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:16.360667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513be42-5b11-4bff-8c5a-ec226a42b307 admin admin] PATH_INFO: `/v3/projects/e8c500936623419bb21a24d7211d9126` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:16.369984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513be42-5b11-4bff-8c5a-ec226a42b307 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8c500936623419bb21a24d7211d9126)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:16.369984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513be42-5b11-4bff-8c5a-ec226a42b307 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:16.519913 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:22:16.524026 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c513be42-5b11-4bff-8c5a-ec226a42b307 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8c500936623419bb21a24d7211d9126', 'request_id': 'req-c513be42-5b11-4bff-8c5a-ec226a42b307'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:16.529103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c513be42-5b11-4bff-8c5a-ec226a42b307 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8c500936623419bb21a24d7211d9126', 'request_id': 'req-c513be42-5b11-4bff-8c5a-ec226a42b307'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:16.536732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c513be42-5b11-4bff-8c5a-ec226a42b307 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8c500936623419bb21a24d7211d9126', 'request_id': 'req-c513be42-5b11-4bff-8c5a-ec226a42b307'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:16.607775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c513be42-5b11-4bff-8c5a-ec226a42b307 admin admin] The token cache is being invalidate because project e8c500936623419bb21a24d7211d9126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:22:16.607888 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c513be42-5b11-4bff-8c5a-ec226a42b307 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:16.608540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-c513be42-5b11-4bff-8c5a-ec226a42b307 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:22:16.611575 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1087/2207] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:22:16 2022] DELETE /identity/v3/projects/e8c500936623419bb21a24d7211d9126 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:22:16.616215 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b989e75-6a04-47ef-ab27-57b9801b55dd None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:16.616731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b989e75-6a04-47ef-ab27-57b9801b55dd None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:16.618447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b989e75-6a04-47ef-ab27-57b9801b55dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:16.644232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9b989e75-6a04-47ef-ab27-57b9801b55dd None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:16.645524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9b989e75-6a04-47ef-ab27-57b9801b55dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:16.693198 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:22:16.732301 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1121/2208] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:16.735326 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcd73e-496b-48af-af2b-af2ef80e9040 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:16.736043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcd73e-496b-48af-af2b-af2ef80e9040 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:16.736450 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcd73e-496b-48af-af2b-af2ef80e9040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:16.757702 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-17bcd73e-496b-48af-af2b-af2ef80e9040 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:16.759147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-17bcd73e-496b-48af-af2b-af2ef80e9040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:16.771170 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1088/2209] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:16.775729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b989e75-6a04-47ef-ab27-57b9801b55dd None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:16.787425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b989e75-6a04-47ef-ab27-57b9801b55dd None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:16.792241 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a375b-ffe1-49ef-b09a-d193e5ab6541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:16.794441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a375b-ffe1-49ef-b09a-d193e5ab6541 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:16.794441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a375b-ffe1-49ef-b09a-d193e5ab6541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:16.794441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a375b-ffe1-49ef-b09a-d193e5ab6541 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:16.794441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a375b-ffe1-49ef-b09a-d193e5ab6541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:22:16.794441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a375b-ffe1-49ef-b09a-d193e5ab6541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:16.795926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a375b-ffe1-49ef-b09a-d193e5ab6541 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:16.809060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1122/2210] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:22:16.813307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcd73e-496b-48af-af2b-af2ef80e9040 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:16.822758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcd73e-496b-48af-af2b-af2ef80e9040 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:16.827373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266427d6-ce4d-4037-8c9d-88a78647a615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:16.827960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266427d6-ce4d-4037-8c9d-88a78647a615 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:16.828238 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266427d6-ce4d-4037-8c9d-88a78647a615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:16.828535 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266427d6-ce4d-4037-8c9d-88a78647a615 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:16.829233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266427d6-ce4d-4037-8c9d-88a78647a615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:16.831916 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266427d6-ce4d-4037-8c9d-88a78647a615 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:16.932096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1089/2211] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:22:16 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:22:16.933296 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a375b-ffe1-49ef-b09a-d193e5ab6541 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:16.943743 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a375b-ffe1-49ef-b09a-d193e5ab6541 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:17.002696 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fbd42-ddc4-4cc5-bee7-e932aa80b3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:17.003318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fbd42-ddc4-4cc5-bee7-e932aa80b3cd admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:17.003596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fbd42-ddc4-4cc5-bee7-e932aa80b3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:17.003870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fbd42-ddc4-4cc5-bee7-e932aa80b3cd admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:17.004494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fbd42-ddc4-4cc5-bee7-e932aa80b3cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:17.006505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fbd42-ddc4-4cc5-bee7-e932aa80b3cd admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:17.130745 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1123/2212] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:22:16 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:22:17.132092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266427d6-ce4d-4037-8c9d-88a78647a615 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:17.143230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266427d6-ce4d-4037-8c9d-88a78647a615 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:17.147409 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c327f-4f68-4ddd-835a-a42d6a1d5847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:17.148060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c327f-4f68-4ddd-835a-a42d6a1d5847 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:17.148346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c327f-4f68-4ddd-835a-a42d6a1d5847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:17.148620 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c327f-4f68-4ddd-835a-a42d6a1d5847 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:17.149315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c327f-4f68-4ddd-835a-a42d6a1d5847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:17.150910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c327f-4f68-4ddd-835a-a42d6a1d5847 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:17.162555 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1090/2213] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:22:17.166999 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fbd42-ddc4-4cc5-bee7-e932aa80b3cd admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:17.178389 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fbd42-ddc4-4cc5-bee7-e932aa80b3cd admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:17.183244 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7390161-9cc7-4c05-82ff-f2fb31a5d9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:17.183787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7390161-9cc7-4c05-82ff-f2fb31a5d9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:17.184578 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7390161-9cc7-4c05-82ff-f2fb31a5d9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:17.184578 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7390161-9cc7-4c05-82ff-f2fb31a5d9e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:17.184999 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7390161-9cc7-4c05-82ff-f2fb31a5d9e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:17.190776 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7390161-9cc7-4c05-82ff-f2fb31a5d9e7 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:17.199993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1124/2214] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:22:17.203850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c327f-4f68-4ddd-835a-a42d6a1d5847 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:17.214703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c327f-4f68-4ddd-835a-a42d6a1d5847 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:17.220144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9ec1a-5ea2-4816-850c-0093d81847a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:17.221332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9ec1a-5ea2-4816-850c-0093d81847a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:17.228447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9ec1a-5ea2-4816-850c-0093d81847a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:17.228447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9ec1a-5ea2-4816-850c-0093d81847a1 admin admin] PATH_INFO: `/v3/projects/478411e9884545398aa7fbec4279a57b/users/fbd28b2571a540e7bdd9c15684dee82d/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:17.240183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9ec1a-5ea2-4816-850c-0093d81847a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=478411e9884545398aa7fbec4279a57b, user_id=fbd28b2571a540e7bdd9c15684dee82d, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:17.242899 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9ec1a-5ea2-4816-850c-0093d81847a1 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:17.315165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1091/2215] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:22:17 2022] PUT /identity/v3/projects/478411e9884545398aa7fbec4279a57b/users/fbd28b2571a540e7bdd9c15684dee82d/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:22:17.320836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a0b3d-4cf3-4692-beb5-a5221088ffe0 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:17.321137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a0b3d-4cf3-4692-beb5-a5221088ffe0 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:17.321421 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a0b3d-4cf3-4692-beb5-a5221088ffe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:17.344745 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-174a0b3d-4cf3-4692-beb5-a5221088ffe0 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:17.345427 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-174a0b3d-4cf3-4692-beb5-a5221088ffe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:17.409725 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1125/2216] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:17.419651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9ec1a-5ea2-4816-850c-0093d81847a1 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:17.421551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f0f9ec1a-5ea2-4816-850c-0093d81847a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:17.483798 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9ec1a-5ea2-4816-850c-0093d81847a1 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:17.488292 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5e2aa-0aec-4dd4-929a-708bf76d011b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:17.488993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5e2aa-0aec-4dd4-929a-708bf76d011b service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:17.489234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5e2aa-0aec-4dd4-929a-708bf76d011b service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:17.489508 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5e2aa-0aec-4dd4-929a-708bf76d011b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:17.498998 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5e2aa-0aec-4dd4-929a-708bf76d011b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:17.500796 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5e2aa-0aec-4dd4-929a-708bf76d011b service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:17.511111 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1092/2217] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:22:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:22:18.986450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb019c-f0b0-4088-9f82-bb36b2803d3c None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:18.986959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb019c-f0b0-4088-9f82-bb36b2803d3c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:18.990596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb019c-f0b0-4088-9f82-bb36b2803d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:19.006237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-35eb019c-f0b0-4088-9f82-bb36b2803d3c None None] MFA Rules not processed for user `099b87a6c6b8409ab188c0b71ded7f55`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:19.007438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-35eb019c-f0b0-4088-9f82-bb36b2803d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:19.064297 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:22:19.078232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1126/2218] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:22:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:23.078379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61170fe-434c-4930-b5a0-5ee449a2ffaa None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:23.078958 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61170fe-434c-4930-b5a0-5ee449a2ffaa None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:23.079241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61170fe-434c-4930-b5a0-5ee449a2ffaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:23.117899 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-f61170fe-434c-4930-b5a0-5ee449a2ffaa None None] MFA Rules not processed for user `fbd28b2571a540e7bdd9c15684dee82d`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:23.119725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f61170fe-434c-4930-b5a0-5ee449a2ffaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:23.198151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1093/2219] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:23.202474 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb019c-f0b0-4088-9f82-bb36b2803d3c None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:23.214106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb019c-f0b0-4088-9f82-bb36b2803d3c None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:23.220712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df278c-ddb0-46ff-814f-23b47d67c2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:23.221699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df278c-ddb0-46ff-814f-23b47d67c2ce admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:23.222049 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df278c-ddb0-46ff-814f-23b47d67c2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:23.222526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df278c-ddb0-46ff-814f-23b47d67c2ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:23.223254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df278c-ddb0-46ff-814f-23b47d67c2ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:23.225529 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df278c-ddb0-46ff-814f-23b47d67c2ce admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:23.322019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1127/2220] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:22:23 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:22:23.329290 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61170fe-434c-4930-b5a0-5ee449a2ffaa None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:23.337772 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:22:23.346418 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61170fe-434c-4930-b5a0-5ee449a2ffaa None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:23.430599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a74e9-1f55-4b69-8322-6c1dd90776e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:23.431236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a74e9-1f55-4b69-8322-6c1dd90776e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:23.431519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a74e9-1f55-4b69-8322-6c1dd90776e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:23.431793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a74e9-1f55-4b69-8322-6c1dd90776e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:23.432453 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a74e9-1f55-4b69-8322-6c1dd90776e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:23.435173 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a74e9-1f55-4b69-8322-6c1dd90776e3 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:23.546208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1094/2221] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:22:23 2022] POST /identity/v3/users => generated 299 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:22:23.549116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df278c-ddb0-46ff-814f-23b47d67c2ce admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:23.584205 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df278c-ddb0-46ff-814f-23b47d67c2ce admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:23.591945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056646b-d081-4da8-8229-c7c20d2c7884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:23.592630 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056646b-d081-4da8-8229-c7c20d2c7884 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:23.593055 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056646b-d081-4da8-8229-c7c20d2c7884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:23.593357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056646b-d081-4da8-8229-c7c20d2c7884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:23.594086 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056646b-d081-4da8-8229-c7c20d2c7884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:23.596236 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056646b-d081-4da8-8229-c7c20d2c7884 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:23.606993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1128/2222] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:22:23.611327 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a74e9-1f55-4b69-8322-6c1dd90776e3 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:23.635134 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a74e9-1f55-4b69-8322-6c1dd90776e3 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:23.652681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e90550-31a7-42c2-903d-75b99221b784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:23.654260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e90550-31a7-42c2-903d-75b99221b784 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:23.654758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e90550-31a7-42c2-903d-75b99221b784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:23.655163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e90550-31a7-42c2-903d-75b99221b784 admin admin] PATH_INFO: `/v3/projects/32c35b5151f54415a42230e5350c95dc/users/7bd00413b8d04ebbbe093cd16b63a25f/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:23.675989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e90550-31a7-42c2-903d-75b99221b784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32c35b5151f54415a42230e5350c95dc, user_id=7bd00413b8d04ebbbe093cd16b63a25f, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:23.678980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e90550-31a7-42c2-903d-75b99221b784 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:23.787543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1095/2223] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:22:23 2022] PUT /identity/v3/projects/32c35b5151f54415a42230e5350c95dc/users/7bd00413b8d04ebbbe093cd16b63a25f/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:22:23.791589 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056646b-d081-4da8-8229-c7c20d2c7884 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:23.814553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056646b-d081-4da8-8229-c7c20d2c7884 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:23.890928 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3958826-9523-4053-8b79-27d81b418229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:23.891881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3958826-9523-4053-8b79-27d81b418229 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:23.892206 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3958826-9523-4053-8b79-27d81b418229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:23.892623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3958826-9523-4053-8b79-27d81b418229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:23.893364 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3958826-9523-4053-8b79-27d81b418229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:23.895175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3958826-9523-4053-8b79-27d81b418229 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:23.912527 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e90550-31a7-42c2-903d-75b99221b784 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:23.924616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e90550-31a7-42c2-903d-75b99221b784 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:23.924836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1129/2224] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:22:23.930365 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5e9c9-f511-4489-ab86-be8705f484c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:23.937160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5e9c9-f511-4489-ab86-be8705f484c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:23.937160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5e9c9-f511-4489-ab86-be8705f484c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:23.937160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5e9c9-f511-4489-ab86-be8705f484c5 admin admin] PATH_INFO: `/v3/projects/32c35b5151f54415a42230e5350c95dc/users/7bd00413b8d04ebbbe093cd16b63a25f/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:23.937160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5e9c9-f511-4489-ab86-be8705f484c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32c35b5151f54415a42230e5350c95dc, user_id=7bd00413b8d04ebbbe093cd16b63a25f, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:23.937160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5e9c9-f511-4489-ab86-be8705f484c5 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:23.948451 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1096/2225] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:22:23 2022] PUT /identity/v3/projects/32c35b5151f54415a42230e5350c95dc/users/7bd00413b8d04ebbbe093cd16b63a25f/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:22:30.786648 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b802f-173a-470d-99a0-11993f26fd75 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:30.787127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b802f-173a-470d-99a0-11993f26fd75 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:30.787221 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b802f-173a-470d-99a0-11993f26fd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:30.822391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-584b802f-173a-470d-99a0-11993f26fd75 None None] MFA Rules not processed for user `7bd00413b8d04ebbbe093cd16b63a25f`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:30.824196 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-584b802f-173a-470d-99a0-11993f26fd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:30.901825 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1130/2226] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:22:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:30.914913 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5e9c9-f511-4489-ab86-be8705f484c5 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:30.916780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-dfe5e9c9-f511-4489-ab86-be8705f484c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:30.985923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5e9c9-f511-4489-ab86-be8705f484c5 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:30.992347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629698de-eb66-4980-8e70-b7348d8da6b5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe9f3fec7104a289e05e88e2d751458', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:30.992347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629698de-eb66-4980-8e70-b7348d8da6b5 service glance] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:30.992347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629698de-eb66-4980-8e70-b7348d8da6b5 service glance] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:30.992347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629698de-eb66-4980-8e70-b7348d8da6b5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:31.060524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629698de-eb66-4980-8e70-b7348d8da6b5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:31.062011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629698de-eb66-4980-8e70-b7348d8da6b5 service glance] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:31.071771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1097/2227] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:22:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:22:31.442161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e9723-42bd-43f7-90da-cee38f5a1f51 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:31.451813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e9723-42bd-43f7-90da-cee38f5a1f51 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:31.451813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e9723-42bd-43f7-90da-cee38f5a1f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:31.469613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-0e8e9723-42bd-43f7-90da-cee38f5a1f51 None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:31.471128 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0e8e9723-42bd-43f7-90da-cee38f5a1f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:31.527430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:22:31.554734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1131/2228] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Aug 12 12:22:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:31.604396 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629698de-eb66-4980-8e70-b7348d8da6b5 service glance] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:31.606098 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-629698de-eb66-4980-8e70-b7348d8da6b5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:31.621346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:22:31.711774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629698de-eb66-4980-8e70-b7348d8da6b5 service glance] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:31.717231 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b778d7-c93d-47a1-981c-271b73777309 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:31.717843 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b778d7-c93d-47a1-981c-271b73777309 service swift] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:31.718140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b778d7-c93d-47a1-981c-271b73777309 service swift] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:31.718424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b778d7-c93d-47a1-981c-271b73777309 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:31.727913 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b778d7-c93d-47a1-981c-271b73777309 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:31.729555 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b778d7-c93d-47a1-981c-271b73777309 service swift] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:31.738161 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1098/2229] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:22:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 12:22:35.264084 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df249e0-583e-44de-9415-64c2fe38f09e None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:35.264593 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df249e0-583e-44de-9415-64c2fe38f09e None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:35.264729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df249e0-583e-44de-9415-64c2fe38f09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:35.292700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9df249e0-583e-44de-9415-64c2fe38f09e None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:35.294238 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9df249e0-583e-44de-9415-64c2fe38f09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:35.303548 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1132/2230] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Aug 12 12:22:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:35.329686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b778d7-c93d-47a1-981c-271b73777309 service swift] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:35.362438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b778d7-c93d-47a1-981c-271b73777309 service swift] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:35.368521 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aac449-3cf4-4e04-a951-8659ae4bc577 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:35.369312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aac449-3cf4-4e04-a951-8659ae4bc577 service swift] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:35.369720 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aac449-3cf4-4e04-a951-8659ae4bc577 service swift] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:35.370100 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aac449-3cf4-4e04-a951-8659ae4bc577 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:35.382176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aac449-3cf4-4e04-a951-8659ae4bc577 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:35.388264 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aac449-3cf4-4e04-a951-8659ae4bc577 service swift] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:35.401685 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1099/2231] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:22:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 12:22:44.674367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa84dc-d0d3-4eed-bb46-c80e2d2acd25 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:44.674932 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa84dc-d0d3-4eed-bb46-c80e2d2acd25 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:44.674932 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa84dc-d0d3-4eed-bb46-c80e2d2acd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:44.698586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-60aa84dc-d0d3-4eed-bb46-c80e2d2acd25 None None] MFA Rules not processed for user `fbd28b2571a540e7bdd9c15684dee82d`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:44.700020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-60aa84dc-d0d3-4eed-bb46-c80e2d2acd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:44.707551 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1133/2232] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:22:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:45.395435 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=33 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:22:54.692855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=13 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:22:54.973932 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73908134-17b0-4d12-ab8c-15c94d20306e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:54.974395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73908134-17b0-4d12-ab8c-15c94d20306e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:54.974518 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73908134-17b0-4d12-ab8c-15c94d20306e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:54.994977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-73908134-17b0-4d12-ab8c-15c94d20306e None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:54.996185 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-73908134-17b0-4d12-ab8c-15c94d20306e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:55.003767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1100/2233] 10.222.0.27 () {58 vars in 967 bytes} [Fri Aug 12 12:22:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:55.016311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa84dc-d0d3-4eed-bb46-c80e2d2acd25 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:55.029074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa84dc-d0d3-4eed-bb46-c80e2d2acd25 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:55.035821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd18c2-47fa-4109-9d6a-4aa035fb80a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:55.042609 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd18c2-47fa-4109-9d6a-4aa035fb80a6 service swift] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:55.044458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd18c2-47fa-4109-9d6a-4aa035fb80a6 service swift] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:55.044458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd18c2-47fa-4109-9d6a-4aa035fb80a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:55.055472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd18c2-47fa-4109-9d6a-4aa035fb80a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:55.057322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd18c2-47fa-4109-9d6a-4aa035fb80a6 service swift] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:55.066568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1134/2234] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:22:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 12:22:55.479387 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefee66-025c-41a5-8019-c1958b2ab555 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:55.479387 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefee66-025c-41a5-8019-c1958b2ab555 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:55.479387 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daefee66-025c-41a5-8019-c1958b2ab555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:55.498031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-daefee66-025c-41a5-8019-c1958b2ab555 None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:55.499260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-daefee66-025c-41a5-8019-c1958b2ab555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:55.507686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1101/2235] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Aug 12 12:22:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:55.521174 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd18c2-47fa-4109-9d6a-4aa035fb80a6 service swift] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:55.537078 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd18c2-47fa-4109-9d6a-4aa035fb80a6 service swift] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:55.544489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bb1f1-7ee9-4ef3-bee5-6347a546e4ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:55.545377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bb1f1-7ee9-4ef3-bee5-6347a546e4ae service swift] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:55.545741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bb1f1-7ee9-4ef3-bee5-6347a546e4ae service swift] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:55.546716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bb1f1-7ee9-4ef3-bee5-6347a546e4ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:55.559229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366bb1f1-7ee9-4ef3-bee5-6347a546e4ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:55.561149 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366bb1f1-7ee9-4ef3-bee5-6347a546e4ae service swift] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:55.571020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1135/2236] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:22:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 12:22:56.702386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a8d26-b50b-482b-841e-d4e931721b48 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:56.702869 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a8d26-b50b-482b-841e-d4e931721b48 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:56.702977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a8d26-b50b-482b-841e-d4e931721b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:56.728045 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-d58a8d26-b50b-482b-841e-d4e931721b48 None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:56.729722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d58a8d26-b50b-482b-841e-d4e931721b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:56.737753 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1102/2237] 10.222.0.27 () {58 vars in 967 bytes} [Fri Aug 12 12:22:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:56.749681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bb1f1-7ee9-4ef3-bee5-6347a546e4ae service swift] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:56.761691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bb1f1-7ee9-4ef3-bee5-6347a546e4ae service swift] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:56.766867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec26db8-010a-47e3-a2ff-09b5591d9945 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:56.767468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec26db8-010a-47e3-a2ff-09b5591d9945 service swift] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:56.771227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec26db8-010a-47e3-a2ff-09b5591d9945 service swift] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:56.771227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec26db8-010a-47e3-a2ff-09b5591d9945 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:56.777657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec26db8-010a-47e3-a2ff-09b5591d9945 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:56.779239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec26db8-010a-47e3-a2ff-09b5591d9945 service swift] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:56.794108 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1136/2238] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:22:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 12:22:57.411107 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb2532-5f96-4e15-a5c9-9e147306ccfc None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:57.411396 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb2532-5f96-4e15-a5c9-9e147306ccfc None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:57.411675 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb2532-5f96-4e15-a5c9-9e147306ccfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:57.431713 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-16eb2532-5f96-4e15-a5c9-9e147306ccfc None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:57.433080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-16eb2532-5f96-4e15-a5c9-9e147306ccfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:57.440952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1103/2239] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Aug 12 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:57.452688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec26db8-010a-47e3-a2ff-09b5591d9945 service swift] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:57.463410 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec26db8-010a-47e3-a2ff-09b5591d9945 service swift] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:57.470652 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f4083-0f55-485f-8398-817a3709b07c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:57.470652 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f4083-0f55-485f-8398-817a3709b07c service swift] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:57.470652 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f4083-0f55-485f-8398-817a3709b07c service swift] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:57.470652 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f4083-0f55-485f-8398-817a3709b07c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:57.477890 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f4083-0f55-485f-8398-817a3709b07c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:57.479466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f4083-0f55-485f-8398-817a3709b07c service swift] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:57.487890 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1137/2240] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:22:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 12:22:58.106628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b961e-aa23-4326-a522-f2676581d570 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:58.107113 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b961e-aa23-4326-a522-f2676581d570 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:58.107205 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b961e-aa23-4326-a522-f2676581d570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:58.126857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-ce0b961e-aa23-4326-a522-f2676581d570 None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:22:58.128928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-ce0b961e-aa23-4326-a522-f2676581d570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:22:58.136336 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1104/2241] 10.222.0.27 () {58 vars in 967 bytes} [Fri Aug 12 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:22:58.148440 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f4083-0f55-485f-8398-817a3709b07c service swift] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:22:58.160039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f4083-0f55-485f-8398-817a3709b07c service swift] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:22:58.165274 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ac88b-0b88-4a24-b227-cf7acd061f04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:22:58.165879 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ac88b-0b88-4a24-b227-cf7acd061f04 service swift] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:22:58.166165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ac88b-0b88-4a24-b227-cf7acd061f04 service swift] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:22:58.166439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ac88b-0b88-4a24-b227-cf7acd061f04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:22:58.174530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ac88b-0b88-4a24-b227-cf7acd061f04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:22:58.177003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ac88b-0b88-4a24-b227-cf7acd061f04 service swift] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:22:58.184992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1138/2242] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:22:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 12:23:00.161466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf4fb5-03e1-4f4d-8175-e09f39b67562 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:00.161959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf4fb5-03e1-4f4d-8175-e09f39b67562 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:00.162043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf4fb5-03e1-4f4d-8175-e09f39b67562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:00.190523 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-adcf4fb5-03e1-4f4d-8175-e09f39b67562 None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:23:00.193024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-adcf4fb5-03e1-4f4d-8175-e09f39b67562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:00.205560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1105/2243] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Aug 12 12:23:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:23:00.214789 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ac88b-0b88-4a24-b227-cf7acd061f04 service swift] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:00.226601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ac88b-0b88-4a24-b227-cf7acd061f04 service swift] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:00.232103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97878c83-9738-4b5c-a390-8ddd7ee99ce3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:00.235850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97878c83-9738-4b5c-a390-8ddd7ee99ce3 service swift] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:00.235850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97878c83-9738-4b5c-a390-8ddd7ee99ce3 service swift] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:00.235850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97878c83-9738-4b5c-a390-8ddd7ee99ce3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:00.243503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97878c83-9738-4b5c-a390-8ddd7ee99ce3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:00.245147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97878c83-9738-4b5c-a390-8ddd7ee99ce3 service swift] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:00.255219 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1139/2244] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:23:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 12 12:23:01.899754 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf4fb5-03e1-4f4d-8175-e09f39b67562 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:01.901205 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-adcf4fb5-03e1-4f4d-8175-e09f39b67562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:01.975063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf4fb5-03e1-4f4d-8175-e09f39b67562 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:01.980123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb549b26-aea7-4de5-8076-add571a3765f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:01.981030 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb549b26-aea7-4de5-8076-add571a3765f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:01.981626 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb549b26-aea7-4de5-8076-add571a3765f admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:01.981626 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb549b26-aea7-4de5-8076-add571a3765f admin admin] PATH_INFO: `/v3/users/013b2068b6e8422781a93b4ecef436bc` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:01.982873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb549b26-aea7-4de5-8076-add571a3765f admin admin] RBAC: Authorizing `identity:delete_user(user_id=013b2068b6e8422781a93b4ecef436bc)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:01.984940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb549b26-aea7-4de5-8076-add571a3765f admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:02.183082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-fb549b26-aea7-4de5-8076-add571a3765f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '013b2068b6e8422781a93b4ecef436bc'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:02.183743 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-fb549b26-aea7-4de5-8076-add571a3765f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '013b2068b6e8422781a93b4ecef436bc'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:02.257727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-fb549b26-aea7-4de5-8076-add571a3765f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '013b2068b6e8422781a93b4ecef436bc'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:02.271325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-fb549b26-aea7-4de5-8076-add571a3765f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '013b2068b6e8422781a93b4ecef436bc'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:02.282906 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-fb549b26-aea7-4de5-8076-add571a3765f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '013b2068b6e8422781a93b4ecef436bc'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:02.290983 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1106/2245] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:23:01 2022] DELETE /identity/v3/users/013b2068b6e8422781a93b4ecef436bc => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:23:02.302506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d62591-a3fc-40b5-9d57-b2e337752561 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:02.303174 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d62591-a3fc-40b5-9d57-b2e337752561 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:02.303174 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d62591-a3fc-40b5-9d57-b2e337752561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:02.328774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-22d62591-a3fc-40b5-9d57-b2e337752561 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:23:02.330077 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-22d62591-a3fc-40b5-9d57-b2e337752561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:02.392874 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1140/2246] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:23:02.406821 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb549b26-aea7-4de5-8076-add571a3765f admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:02.406821 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-fb549b26-aea7-4de5-8076-add571a3765f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:02.458777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:23:02.475907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb549b26-aea7-4de5-8076-add571a3765f admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:02.479658 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f16b5a-640a-445b-9273-b2b72d8531f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:02.480265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f16b5a-640a-445b-9273-b2b72d8531f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:02.480543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f16b5a-640a-445b-9273-b2b72d8531f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:02.481069 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f16b5a-640a-445b-9273-b2b72d8531f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:02.493694 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f16b5a-640a-445b-9273-b2b72d8531f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:02.496131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f16b5a-640a-445b-9273-b2b72d8531f5 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:02.519378 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1107/2247] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:23:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:23:02.953678 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d62591-a3fc-40b5-9d57-b2e337752561 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:02.956255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-22d62591-a3fc-40b5-9d57-b2e337752561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:02.985009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f16b5a-640a-445b-9273-b2b72d8531f5 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:02.986551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-78f16b5a-640a-445b-9273-b2b72d8531f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:02.987399 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d62591-a3fc-40b5-9d57-b2e337752561 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:02.998425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1dd1ad-636f-4a28-9e76-912e1aee64a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:03.001029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dd1ad-636f-4a28-9e76-912e1aee64a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:03.001502 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dd1ad-636f-4a28-9e76-912e1aee64a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:03.002116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dd1ad-636f-4a28-9e76-912e1aee64a1 admin admin] PATH_INFO: `/v3/users/011f5d15687d4a55ac595c8f067d6f1d` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:03.004586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1dd1ad-636f-4a28-9e76-912e1aee64a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=011f5d15687d4a55ac595c8f067d6f1d)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:03.005639 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f16b5a-640a-445b-9273-b2b72d8531f5 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:03.006848 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1dd1ad-636f-4a28-9e76-912e1aee64a1 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:03.011268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de252f-0e5b-4a0e-aaed-d655b79271c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:03.012096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de252f-0e5b-4a0e-aaed-d655b79271c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:03.012537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de252f-0e5b-4a0e-aaed-d655b79271c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:03.012995 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de252f-0e5b-4a0e-aaed-d655b79271c1 admin admin] PATH_INFO: `/v3/projects/58a76d86371c448a8ff0596da2235ba5` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:03.014282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de252f-0e5b-4a0e-aaed-d655b79271c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58a76d86371c448a8ff0596da2235ba5)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:03.016893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de252f-0e5b-4a0e-aaed-d655b79271c1 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:03.171946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e7de252f-0e5b-4a0e-aaed-d655b79271c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58a76d86371c448a8ff0596da2235ba5', 'request_id': 'req-e7de252f-0e5b-4a0e-aaed-d655b79271c1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:03.179083 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e7de252f-0e5b-4a0e-aaed-d655b79271c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58a76d86371c448a8ff0596da2235ba5', 'request_id': 'req-e7de252f-0e5b-4a0e-aaed-d655b79271c1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:03.187475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e7de252f-0e5b-4a0e-aaed-d655b79271c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58a76d86371c448a8ff0596da2235ba5', 'request_id': 'req-e7de252f-0e5b-4a0e-aaed-d655b79271c1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:03.293900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e7de252f-0e5b-4a0e-aaed-d655b79271c1 admin admin] The token cache is being invalidate because project 58a76d86371c448a8ff0596da2235ba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:23:03.294213 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e7de252f-0e5b-4a0e-aaed-d655b79271c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:03.295534 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-e7de252f-0e5b-4a0e-aaed-d655b79271c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:03.298881 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1108/2248] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:23:02 2022] DELETE /identity/v3/projects/58a76d86371c448a8ff0596da2235ba5 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:23:03.305263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57d150-4e21-4090-8469-c4cb26bc8139 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:03.305910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57d150-4e21-4090-8469-c4cb26bc8139 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:03.306338 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57d150-4e21-4090-8469-c4cb26bc8139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:03.312249 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-de1dd1ad-636f-4a28-9e76-912e1aee64a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '011f5d15687d4a55ac595c8f067d6f1d'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:03.328222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-2c57d150-4e21-4090-8469-c4cb26bc8139 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:23:03.329375 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2c57d150-4e21-4090-8469-c4cb26bc8139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:03.415384 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1109/2249] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:23:03.423453 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132f9ce-c2dd-4d91-bf4a-917c4b35e516 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:03.423633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-de1dd1ad-636f-4a28-9e76-912e1aee64a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '011f5d15687d4a55ac595c8f067d6f1d'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:03.423751 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132f9ce-c2dd-4d91-bf4a-917c4b35e516 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:03.424029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132f9ce-c2dd-4d91-bf4a-917c4b35e516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:03.424291 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-de1dd1ad-636f-4a28-9e76-912e1aee64a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '011f5d15687d4a55ac595c8f067d6f1d'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:03.433283 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-de1dd1ad-636f-4a28-9e76-912e1aee64a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '011f5d15687d4a55ac595c8f067d6f1d'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:03.445994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:23:03.447170 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-a132f9ce-c2dd-4d91-bf4a-917c4b35e516 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:23:03.448211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a132f9ce-c2dd-4d91-bf4a-917c4b35e516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:03.448331 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-de1dd1ad-636f-4a28-9e76-912e1aee64a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '011f5d15687d4a55ac595c8f067d6f1d'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:03.454120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1141/2250] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:23:02 2022] DELETE /identity/v3/users/011f5d15687d4a55ac595c8f067d6f1d => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:23:03.456803 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1110/2251] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:23:03.468826 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1dd1ad-636f-4a28-9e76-912e1aee64a1 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:03.572587 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1dd1ad-636f-4a28-9e76-912e1aee64a1 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:03.578878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32991284-a3d1-4134-ac22-5d74dddac804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:03.579435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32991284-a3d1-4134-ac22-5d74dddac804 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:03.579732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32991284-a3d1-4134-ac22-5d74dddac804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:03.579992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32991284-a3d1-4134-ac22-5d74dddac804 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:03.580960 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32991284-a3d1-4134-ac22-5d74dddac804 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:23:03.581213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32991284-a3d1-4134-ac22-5d74dddac804 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:03.583508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32991284-a3d1-4134-ac22-5d74dddac804 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:03.596881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1142/2252] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:23:03.601977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132f9ce-c2dd-4d91-bf4a-917c4b35e516 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:03.612270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132f9ce-c2dd-4d91-bf4a-917c4b35e516 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:03.618673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842b3fd-f423-4280-b9dd-f19cc11e6619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:03.619416 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842b3fd-f423-4280-b9dd-f19cc11e6619 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:03.619774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842b3fd-f423-4280-b9dd-f19cc11e6619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:03.620154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842b3fd-f423-4280-b9dd-f19cc11e6619 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:03.620900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842b3fd-f423-4280-b9dd-f19cc11e6619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:03.631011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842b3fd-f423-4280-b9dd-f19cc11e6619 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:03.790181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32991284-a3d1-4134-ac22-5d74dddac804 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:03.793502 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-32991284-a3d1-4134-ac22-5d74dddac804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:03.801415 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1111/2253] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:23:03 2022] POST /identity/v3/projects => generated 389 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:23:03.811447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842b3fd-f423-4280-b9dd-f19cc11e6619 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:03.832929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32991284-a3d1-4134-ac22-5d74dddac804 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:03.924656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:23:04.002040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842b3fd-f423-4280-b9dd-f19cc11e6619 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:04.008753 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1877c04-4810-4773-8ef2-0cea44f9e890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:04.009649 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1877c04-4810-4773-8ef2-0cea44f9e890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:04.010029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1877c04-4810-4773-8ef2-0cea44f9e890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:04.010440 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1877c04-4810-4773-8ef2-0cea44f9e890 admin admin] PATH_INFO: `/v3/projects/8164da757959471cbbfb3b51af1edebd` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:04.012158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1877c04-4810-4773-8ef2-0cea44f9e890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8164da757959471cbbfb3b51af1edebd)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:04.014350 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1877c04-4810-4773-8ef2-0cea44f9e890 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:04.036622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47581f4-a925-49d4-8dcd-67ac9f2825d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:04.037646 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47581f4-a925-49d4-8dcd-67ac9f2825d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:04.039241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47581f4-a925-49d4-8dcd-67ac9f2825d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:04.039621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47581f4-a925-49d4-8dcd-67ac9f2825d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:04.040778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47581f4-a925-49d4-8dcd-67ac9f2825d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:04.043536 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47581f4-a925-49d4-8dcd-67ac9f2825d9 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:04.145481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b1877c04-4810-4773-8ef2-0cea44f9e890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8164da757959471cbbfb3b51af1edebd', 'request_id': 'req-b1877c04-4810-4773-8ef2-0cea44f9e890'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:04.150991 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b1877c04-4810-4773-8ef2-0cea44f9e890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8164da757959471cbbfb3b51af1edebd', 'request_id': 'req-b1877c04-4810-4773-8ef2-0cea44f9e890'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:04.187472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1112/2254] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:23:03 2022] POST /identity/v3/users => generated 295 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:23:04.188502 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b1877c04-4810-4773-8ef2-0cea44f9e890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8164da757959471cbbfb3b51af1edebd', 'request_id': 'req-b1877c04-4810-4773-8ef2-0cea44f9e890'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:04.194894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47581f4-a925-49d4-8dcd-67ac9f2825d9 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:04.206256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47581f4-a925-49d4-8dcd-67ac9f2825d9 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:04.275045 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94408d0-16a8-4c04-b2f2-308aa166673a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:04.275699 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94408d0-16a8-4c04-b2f2-308aa166673a admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:04.275980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94408d0-16a8-4c04-b2f2-308aa166673a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:04.276273 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94408d0-16a8-4c04-b2f2-308aa166673a admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:04.276965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94408d0-16a8-4c04-b2f2-308aa166673a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:04.278623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94408d0-16a8-4c04-b2f2-308aa166673a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:04.293153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1113/2255] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:23:04.297464 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94408d0-16a8-4c04-b2f2-308aa166673a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:04.313412 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94408d0-16a8-4c04-b2f2-308aa166673a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:04.324694 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb201e4-5c42-4829-8a27-52a00641d7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:04.328595 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb201e4-5c42-4829-8a27-52a00641d7db admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:04.328595 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb201e4-5c42-4829-8a27-52a00641d7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:04.328595 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb201e4-5c42-4829-8a27-52a00641d7db admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:04.328595 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb201e4-5c42-4829-8a27-52a00641d7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:04.328974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b1877c04-4810-4773-8ef2-0cea44f9e890 admin admin] The token cache is being invalidate because project 8164da757959471cbbfb3b51af1edebd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:23:04.328974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b1877c04-4810-4773-8ef2-0cea44f9e890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:04.329542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb201e4-5c42-4829-8a27-52a00641d7db admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:04.329678 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b1877c04-4810-4773-8ef2-0cea44f9e890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:04.331043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1143/2256] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:23:03 2022] DELETE /identity/v3/projects/8164da757959471cbbfb3b51af1edebd => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:23:04.341641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1114/2257] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:23:04.346035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1877c04-4810-4773-8ef2-0cea44f9e890 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:04.347568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b1877c04-4810-4773-8ef2-0cea44f9e890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:04.380345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1877c04-4810-4773-8ef2-0cea44f9e890 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:04.387428 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02500a85-4c08-48dd-99d2-1c6926980b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:04.388113 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02500a85-4c08-48dd-99d2-1c6926980b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:04.388488 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02500a85-4c08-48dd-99d2-1c6926980b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:04.388805 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02500a85-4c08-48dd-99d2-1c6926980b92 admin admin] PATH_INFO: `/v3/projects/bcd1b1ee6d5f44808d600e5d67fa2733/users/69a6e40145cc4d62b20888685f8f8fdf/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:04.408348 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02500a85-4c08-48dd-99d2-1c6926980b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcd1b1ee6d5f44808d600e5d67fa2733, user_id=69a6e40145cc4d62b20888685f8f8fdf, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:04.412758 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02500a85-4c08-48dd-99d2-1c6926980b92 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:04.416027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:23:04.490454 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1144/2258] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:23:04 2022] PUT /identity/v3/projects/bcd1b1ee6d5f44808d600e5d67fa2733/users/69a6e40145cc4d62b20888685f8f8fdf/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:23:04.502420 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af2fdc-b791-468b-94d6-77bb21e29de9 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:04.502710 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af2fdc-b791-468b-94d6-77bb21e29de9 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:04.502984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af2fdc-b791-468b-94d6-77bb21e29de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:04.541063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-14af2fdc-b791-468b-94d6-77bb21e29de9 None None] MFA Rules not processed for user `69a6e40145cc4d62b20888685f8f8fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:23:04.542345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-14af2fdc-b791-468b-94d6-77bb21e29de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:04.608742 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1115/2259] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:23:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:23:04.620571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02500a85-4c08-48dd-99d2-1c6926980b92 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:04.622338 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-02500a85-4c08-48dd-99d2-1c6926980b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:04.683633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02500a85-4c08-48dd-99d2-1c6926980b92 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:04.687386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed5243-3c91-42b2-87a5-1dacea887ed3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:04.687996 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed5243-3c91-42b2-87a5-1dacea887ed3 service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:04.688284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed5243-3c91-42b2-87a5-1dacea887ed3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:04.688562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed5243-3c91-42b2-87a5-1dacea887ed3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:04.698703 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eed5243-3c91-42b2-87a5-1dacea887ed3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:04.700316 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eed5243-3c91-42b2-87a5-1dacea887ed3 service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:04.711066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1145/2260] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:23:04 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:23:14.605674 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:23:14.706101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:23:35.135143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af2fdc-b791-468b-94d6-77bb21e29de9 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:35.220272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af2fdc-b791-468b-94d6-77bb21e29de9 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:35.224811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d513c89-407e-452b-8434-774a2141e3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:35.225435 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d513c89-407e-452b-8434-774a2141e3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:35.225721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d513c89-407e-452b-8434-774a2141e3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:35.226076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d513c89-407e-452b-8434-774a2141e3b9 admin admin] PATH_INFO: `/v3/users/69a6e40145cc4d62b20888685f8f8fdf` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:35.227205 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d513c89-407e-452b-8434-774a2141e3b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69a6e40145cc4d62b20888685f8f8fdf)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:35.229692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d513c89-407e-452b-8434-774a2141e3b9 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:35.422126 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6d513c89-407e-452b-8434-774a2141e3b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69a6e40145cc4d62b20888685f8f8fdf'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:35.423147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6d513c89-407e-452b-8434-774a2141e3b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69a6e40145cc4d62b20888685f8f8fdf'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:35.493216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6d513c89-407e-452b-8434-774a2141e3b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69a6e40145cc4d62b20888685f8f8fdf'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:35.506448 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6d513c89-407e-452b-8434-774a2141e3b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69a6e40145cc4d62b20888685f8f8fdf'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:35.514207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6d513c89-407e-452b-8434-774a2141e3b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69a6e40145cc4d62b20888685f8f8fdf'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:35.520136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1116/2261] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:23:35 2022] DELETE /identity/v3/users/69a6e40145cc4d62b20888685f8f8fdf => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:23:35.524659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a92c9-9aff-4302-a282-2a49b5d1e8fe None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:35.525117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a92c9-9aff-4302-a282-2a49b5d1e8fe None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:35.525209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a92c9-9aff-4302-a282-2a49b5d1e8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:35.545278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-709a92c9-9aff-4302-a282-2a49b5d1e8fe None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:23:35.546487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-709a92c9-9aff-4302-a282-2a49b5d1e8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:35.605729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1146/2262] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:23:35.615487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d513c89-407e-452b-8434-774a2141e3b9 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:35.616934 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6d513c89-407e-452b-8434-774a2141e3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:35.673759 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d513c89-407e-452b-8434-774a2141e3b9 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:35.677364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfc28c-cd5b-4bd5-a6a1-caf71286ca51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:35.677961 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfc28c-cd5b-4bd5-a6a1-caf71286ca51 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:35.678243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfc28c-cd5b-4bd5-a6a1-caf71286ca51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:35.678517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfc28c-cd5b-4bd5-a6a1-caf71286ca51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:35.687842 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdfc28c-cd5b-4bd5-a6a1-caf71286ca51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:35.689283 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdfc28c-cd5b-4bd5-a6a1-caf71286ca51 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:35.699281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1117/2263] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:23:36.113529 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a92c9-9aff-4302-a282-2a49b5d1e8fe None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:36.114940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-709a92c9-9aff-4302-a282-2a49b5d1e8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:36.128002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a92c9-9aff-4302-a282-2a49b5d1e8fe None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:36.133610 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7bafb-fed6-4b08-a039-053c79709001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:36.134632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7bafb-fed6-4b08-a039-053c79709001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:36.134914 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7bafb-fed6-4b08-a039-053c79709001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:36.135187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7bafb-fed6-4b08-a039-053c79709001 admin admin] PATH_INFO: `/v3/projects/bcd1b1ee6d5f44808d600e5d67fa2733` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:36.136567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7bafb-fed6-4b08-a039-053c79709001 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcd1b1ee6d5f44808d600e5d67fa2733)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:36.139409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7bafb-fed6-4b08-a039-053c79709001 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:36.245868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d9d7bafb-fed6-4b08-a039-053c79709001 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcd1b1ee6d5f44808d600e5d67fa2733', 'request_id': 'req-d9d7bafb-fed6-4b08-a039-053c79709001'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:36.251667 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d9d7bafb-fed6-4b08-a039-053c79709001 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcd1b1ee6d5f44808d600e5d67fa2733', 'request_id': 'req-d9d7bafb-fed6-4b08-a039-053c79709001'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:36.258063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d9d7bafb-fed6-4b08-a039-053c79709001 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcd1b1ee6d5f44808d600e5d67fa2733', 'request_id': 'req-d9d7bafb-fed6-4b08-a039-053c79709001'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:36.346729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d9d7bafb-fed6-4b08-a039-053c79709001 admin admin] The token cache is being invalidate because project bcd1b1ee6d5f44808d600e5d67fa2733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:23:36.347022 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d9d7bafb-fed6-4b08-a039-053c79709001 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:36.347664 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d9d7bafb-fed6-4b08-a039-053c79709001 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:23:36.349795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1147/2264] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:23:36 2022] DELETE /identity/v3/projects/bcd1b1ee6d5f44808d600e5d67fa2733 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:23:36.355485 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec77346-ec3f-4944-ac7b-8a1b4f5212f0 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:36.356006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec77346-ec3f-4944-ac7b-8a1b4f5212f0 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:36.356187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec77346-ec3f-4944-ac7b-8a1b4f5212f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:36.376152 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-0ec77346-ec3f-4944-ac7b-8a1b4f5212f0 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:23:36.377378 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0ec77346-ec3f-4944-ac7b-8a1b4f5212f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:36.396121 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:23:36.450921 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1118/2265] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:23:36.459046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe9322-cd53-4e24-b949-90b34f3cad61 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:36.459329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe9322-cd53-4e24-b949-90b34f3cad61 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:36.459605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe9322-cd53-4e24-b949-90b34f3cad61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:36.490513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9cbe9322-cd53-4e24-b949-90b34f3cad61 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:23:36.492026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9cbe9322-cd53-4e24-b949-90b34f3cad61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:36.503857 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1148/2266] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:23:36.508284 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec77346-ec3f-4944-ac7b-8a1b4f5212f0 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:36.526878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec77346-ec3f-4944-ac7b-8a1b4f5212f0 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:36.531984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142c9d4-91d1-469f-8a0d-837880204b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:36.532790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142c9d4-91d1-469f-8a0d-837880204b8b admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:36.533117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142c9d4-91d1-469f-8a0d-837880204b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:36.533466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142c9d4-91d1-469f-8a0d-837880204b8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:36.534232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142c9d4-91d1-469f-8a0d-837880204b8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:23:36.534599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142c9d4-91d1-469f-8a0d-837880204b8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:36.536431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142c9d4-91d1-469f-8a0d-837880204b8b admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:36.561388 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1119/2267] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:23:36.566639 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe9322-cd53-4e24-b949-90b34f3cad61 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:36.580345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe9322-cd53-4e24-b949-90b34f3cad61 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:36.595925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27f416-5be2-4662-870a-0f3247d8a2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:36.596691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27f416-5be2-4662-870a-0f3247d8a2d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:36.597271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27f416-5be2-4662-870a-0f3247d8a2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:36.597618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27f416-5be2-4662-870a-0f3247d8a2d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:36.598334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27f416-5be2-4662-870a-0f3247d8a2d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:36.600395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27f416-5be2-4662-870a-0f3247d8a2d3 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:36.702472 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1149/2268] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:23:36 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:23:36.706842 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142c9d4-91d1-469f-8a0d-837880204b8b admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:36.718824 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142c9d4-91d1-469f-8a0d-837880204b8b admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:36.789851 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa55c8-be75-41e9-a18e-847e42083663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:36.790598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa55c8-be75-41e9-a18e-847e42083663 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:36.790941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa55c8-be75-41e9-a18e-847e42083663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:36.791462 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa55c8-be75-41e9-a18e-847e42083663 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:36.792760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa55c8-be75-41e9-a18e-847e42083663 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:36.795337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa55c8-be75-41e9-a18e-847e42083663 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:36.882494 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1120/2269] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:23:36 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:23:36.886407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27f416-5be2-4662-870a-0f3247d8a2d3 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:36.897813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27f416-5be2-4662-870a-0f3247d8a2d3 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:36.903466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3732fa-b1b5-4b44-b761-c0ad5fa7c822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:36.904261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3732fa-b1b5-4b44-b761-c0ad5fa7c822 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:36.904613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3732fa-b1b5-4b44-b761-c0ad5fa7c822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:36.904924 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3732fa-b1b5-4b44-b761-c0ad5fa7c822 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:36.905576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3732fa-b1b5-4b44-b761-c0ad5fa7c822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:36.907181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3732fa-b1b5-4b44-b761-c0ad5fa7c822 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:36.920680 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1150/2270] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:23:36.925376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa55c8-be75-41e9-a18e-847e42083663 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:36.937128 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa55c8-be75-41e9-a18e-847e42083663 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:36.941834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2ffc4-5d10-4d0b-81f1-8ffb11afdca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:36.942438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2ffc4-5d10-4d0b-81f1-8ffb11afdca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:36.942721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2ffc4-5d10-4d0b-81f1-8ffb11afdca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:36.942996 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2ffc4-5d10-4d0b-81f1-8ffb11afdca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:36.943681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2ffc4-5d10-4d0b-81f1-8ffb11afdca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:36.945362 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2ffc4-5d10-4d0b-81f1-8ffb11afdca7 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:36.956458 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1121/2271] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:23:36.960764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3732fa-b1b5-4b44-b761-c0ad5fa7c822 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:36.971419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3732fa-b1b5-4b44-b761-c0ad5fa7c822 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:36.976048 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3cd2f-3ece-4bed-8f8a-6dbee274b6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:36.976726 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3cd2f-3ece-4bed-8f8a-6dbee274b6ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:36.977011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3cd2f-3ece-4bed-8f8a-6dbee274b6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:36.977284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3cd2f-3ece-4bed-8f8a-6dbee274b6ee admin admin] PATH_INFO: `/v3/projects/b20b2b17c8794507a7cd8fecfb50e13b/users/803cf2e2fb664442ae8b0f8919a90aec/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:36.991827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3cd2f-3ece-4bed-8f8a-6dbee274b6ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b20b2b17c8794507a7cd8fecfb50e13b, user_id=803cf2e2fb664442ae8b0f8919a90aec, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:36.994430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3cd2f-3ece-4bed-8f8a-6dbee274b6ee admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:37.065274 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1151/2272] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:23:36 2022] PUT /identity/v3/projects/b20b2b17c8794507a7cd8fecfb50e13b/users/803cf2e2fb664442ae8b0f8919a90aec/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:23:37.070525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b05f4e-1ceb-41c7-a412-a0fac7419615 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:37.070820 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b05f4e-1ceb-41c7-a412-a0fac7419615 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:37.071160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b05f4e-1ceb-41c7-a412-a0fac7419615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:37.093342 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-48b05f4e-1ceb-41c7-a412-a0fac7419615 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:23:37.094645 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-48b05f4e-1ceb-41c7-a412-a0fac7419615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:37.171857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1122/2273] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:23:37.173497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3cd2f-3ece-4bed-8f8a-6dbee274b6ee admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:37.176615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2ef3cd2f-3ece-4bed-8f8a-6dbee274b6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:37.201307 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:23:37.238054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3cd2f-3ece-4bed-8f8a-6dbee274b6ee admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:37.241802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45c73b-5045-4f32-ba89-b252fbde64cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:37.253006 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c73b-5045-4f32-ba89-b252fbde64cb service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:37.253006 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c73b-5045-4f32-ba89-b252fbde64cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:37.253006 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c73b-5045-4f32-ba89-b252fbde64cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:37.253346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45c73b-5045-4f32-ba89-b252fbde64cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:37.254668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45c73b-5045-4f32-ba89-b252fbde64cb service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:37.266486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1152/2274] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:23:41.623873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91217155-054d-4cb0-8e9b-7b8153ceda80 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:41.626127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91217155-054d-4cb0-8e9b-7b8153ceda80 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:41.626127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91217155-054d-4cb0-8e9b-7b8153ceda80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:41.646125 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-91217155-054d-4cb0-8e9b-7b8153ceda80 None None] MFA Rules not processed for user `0156c689e109489d9265dd3dd4d5cfde`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:23:41.647339 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-91217155-054d-4cb0-8e9b-7b8153ceda80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:41.677088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:23:41.715138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1123/2275] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:23:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:23:44.166167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54c5ae-c167-4d42-9b0a-2806d93270d3 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:44.166700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54c5ae-c167-4d42-9b0a-2806d93270d3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:44.166789 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54c5ae-c167-4d42-9b0a-2806d93270d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:44.200589 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-cf54c5ae-c167-4d42-9b0a-2806d93270d3 None None] MFA Rules not processed for user `803cf2e2fb664442ae8b0f8919a90aec`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:23:44.201839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-cf54c5ae-c167-4d42-9b0a-2806d93270d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:44.291191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1153/2276] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:23:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:23:44.297247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91217155-054d-4cb0-8e9b-7b8153ceda80 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:44.319465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91217155-054d-4cb0-8e9b-7b8153ceda80 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:44.323942 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919b804-0261-44ea-9f84-ccc89394491f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:44.324626 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919b804-0261-44ea-9f84-ccc89394491f admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:44.324931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919b804-0261-44ea-9f84-ccc89394491f admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:44.325189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919b804-0261-44ea-9f84-ccc89394491f admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:44.325817 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919b804-0261-44ea-9f84-ccc89394491f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:44.327725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919b804-0261-44ea-9f84-ccc89394491f admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:44.428502 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1124/2277] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:23:44 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:23:44.432765 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf54c5ae-c167-4d42-9b0a-2806d93270d3 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:44.444116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf54c5ae-c167-4d42-9b0a-2806d93270d3 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:44.512562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c1c3a-9cf6-43f6-ad8b-dc9c16c14f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:44.513238 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c1c3a-9cf6-43f6-ad8b-dc9c16c14f8a admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:44.513527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c1c3a-9cf6-43f6-ad8b-dc9c16c14f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:44.513801 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c1c3a-9cf6-43f6-ad8b-dc9c16c14f8a admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:44.514425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c1c3a-9cf6-43f6-ad8b-dc9c16c14f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:44.516971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c1c3a-9cf6-43f6-ad8b-dc9c16c14f8a admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:44.631421 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1154/2278] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:23:44 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:23:44.633513 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919b804-0261-44ea-9f84-ccc89394491f admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:44.645257 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919b804-0261-44ea-9f84-ccc89394491f admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:44.649954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736285bd-23cf-4f37-935d-5b02f09129fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:44.650568 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736285bd-23cf-4f37-935d-5b02f09129fa admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:44.650853 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736285bd-23cf-4f37-935d-5b02f09129fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:44.651131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736285bd-23cf-4f37-935d-5b02f09129fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:44.651840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736285bd-23cf-4f37-935d-5b02f09129fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:44.653945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736285bd-23cf-4f37-935d-5b02f09129fa admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:44.676905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c1c3a-9cf6-43f6-ad8b-dc9c16c14f8a admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:44.682255 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1125/2279] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:23:44.687336 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c1c3a-9cf6-43f6-ad8b-dc9c16c14f8a admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:44.693165 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d95dee-2e09-43ba-b67f-302907da6a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:44.694510 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d95dee-2e09-43ba-b67f-302907da6a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:44.695004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d95dee-2e09-43ba-b67f-302907da6a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:44.695662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d95dee-2e09-43ba-b67f-302907da6a92 admin admin] PATH_INFO: `/v3/projects/0427b62c1d4a4621b1fc8fa8cfa8fea6/users/e8c4df1815df4411b80b4f4152ddedd1/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:44.713729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d95dee-2e09-43ba-b67f-302907da6a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0427b62c1d4a4621b1fc8fa8cfa8fea6, user_id=e8c4df1815df4411b80b4f4152ddedd1, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:44.716864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d95dee-2e09-43ba-b67f-302907da6a92 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:44.795116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1155/2280] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:23:44 2022] PUT /identity/v3/projects/0427b62c1d4a4621b1fc8fa8cfa8fea6/users/e8c4df1815df4411b80b4f4152ddedd1/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:23:44.799332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736285bd-23cf-4f37-935d-5b02f09129fa admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:44.817577 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736285bd-23cf-4f37-935d-5b02f09129fa admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:44.885753 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2757fa-48f3-4e93-a75e-323ef4625f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:44.886368 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2757fa-48f3-4e93-a75e-323ef4625f5f admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:44.886652 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2757fa-48f3-4e93-a75e-323ef4625f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:44.887025 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2757fa-48f3-4e93-a75e-323ef4625f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:44.887780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2757fa-48f3-4e93-a75e-323ef4625f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:44.889614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2757fa-48f3-4e93-a75e-323ef4625f5f admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:44.900336 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1126/2281] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:23:44.904852 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d95dee-2e09-43ba-b67f-302907da6a92 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:44.917654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d95dee-2e09-43ba-b67f-302907da6a92 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:44.923386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af81fc-023f-4ebd-9c18-b99405d44229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:44.924011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af81fc-023f-4ebd-9c18-b99405d44229 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:44.924292 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af81fc-023f-4ebd-9c18-b99405d44229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:44.924562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af81fc-023f-4ebd-9c18-b99405d44229 admin admin] PATH_INFO: `/v3/projects/0427b62c1d4a4621b1fc8fa8cfa8fea6/users/e8c4df1815df4411b80b4f4152ddedd1/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:44.926523 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af81fc-023f-4ebd-9c18-b99405d44229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0427b62c1d4a4621b1fc8fa8cfa8fea6, user_id=e8c4df1815df4411b80b4f4152ddedd1, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:44.929285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af81fc-023f-4ebd-9c18-b99405d44229 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:44.942392 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1156/2282] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:23:44 2022] PUT /identity/v3/projects/0427b62c1d4a4621b1fc8fa8cfa8fea6/users/e8c4df1815df4411b80b4f4152ddedd1/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:23:46.987292 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2757fa-48f3-4e93-a75e-323ef4625f5f admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:47.061250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2757fa-48f3-4e93-a75e-323ef4625f5f admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:47.150280 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b157ee4-1ff1-444f-94be-4629aae6a52f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:47.151659 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b157ee4-1ff1-444f-94be-4629aae6a52f service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:47.152035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b157ee4-1ff1-444f-94be-4629aae6a52f service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:47.152314 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b157ee4-1ff1-444f-94be-4629aae6a52f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:47.154858 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8b157ee4-1ff1-444f-94be-4629aae6a52f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:47.163333 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:23:47.247771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b157ee4-1ff1-444f-94be-4629aae6a52f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:47.249786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b157ee4-1ff1-444f-94be-4629aae6a52f service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:47.277835 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1127/2283] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:23:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:23:48.726822 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af81fc-023f-4ebd-9c18-b99405d44229 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:48.734299 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:23:48.749532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af81fc-023f-4ebd-9c18-b99405d44229 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:48.755216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa48df4-4574-43bc-9cee-6a50caa7fd3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:48.755834 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa48df4-4574-43bc-9cee-6a50caa7fd3f service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:48.756119 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa48df4-4574-43bc-9cee-6a50caa7fd3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:48.756396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa48df4-4574-43bc-9cee-6a50caa7fd3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:48.758449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4aa48df4-4574-43bc-9cee-6a50caa7fd3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:48.779613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa48df4-4574-43bc-9cee-6a50caa7fd3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:48.779613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa48df4-4574-43bc-9cee-6a50caa7fd3f service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:48.791042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1157/2284] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:23:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:23:50.863346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4206b-d955-4da9-bec7-bb17cc1b3256 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:50.863800 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4206b-d955-4da9-bec7-bb17cc1b3256 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:50.864074 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4206b-d955-4da9-bec7-bb17cc1b3256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:50.893702 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-00b4206b-d955-4da9-bec7-bb17cc1b3256 None None] MFA Rules not processed for user `e8c4df1815df4411b80b4f4152ddedd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:23:50.895448 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-00b4206b-d955-4da9-bec7-bb17cc1b3256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:50.983628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1128/2285] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:23:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:23:50.997534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa48df4-4574-43bc-9cee-6a50caa7fd3f service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:50.999662 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4aa48df4-4574-43bc-9cee-6a50caa7fd3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:51.016974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa48df4-4574-43bc-9cee-6a50caa7fd3f service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:51.024784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a726c9-0635-4a4b-990c-e9276138d763 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:51.025863 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a726c9-0635-4a4b-990c-e9276138d763 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:51.026305 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a726c9-0635-4a4b-990c-e9276138d763 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:51.026752 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a726c9-0635-4a4b-990c-e9276138d763 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:51.130273 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a726c9-0635-4a4b-990c-e9276138d763 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:51.132199 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a726c9-0635-4a4b-990c-e9276138d763 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:51.156467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1158/2286] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:23:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:23:53.291450 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc75ca-8f2b-4333-85a7-94edd030ed1e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:53.298087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc75ca-8f2b-4333-85a7-94edd030ed1e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:53.298087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc75ca-8f2b-4333-85a7-94edd030ed1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:53.317811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-5bdc75ca-8f2b-4333-85a7-94edd030ed1e None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:23:53.319115 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-5bdc75ca-8f2b-4333-85a7-94edd030ed1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:53.451921 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1129/2287] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Aug 12 12:23:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:23:53.478196 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a726c9-0635-4a4b-990c-e9276138d763 service nova] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:23:53.479842 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f9a726c9-0635-4a4b-990c-e9276138d763 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:53.586820 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a726c9-0635-4a4b-990c-e9276138d763 service nova] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:23:53.595090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7bf58-a514-4708-a482-1721082459d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:23:53.595739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7bf58-a514-4708-a482-1721082459d0 service swift] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:53.596016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7bf58-a514-4708-a482-1721082459d0 service swift] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:53.596554 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7bf58-a514-4708-a482-1721082459d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:53.616986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7bf58-a514-4708-a482-1721082459d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:23:53.619094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7bf58-a514-4708-a482-1721082459d0 service swift] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:23:53.650815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1159/2288] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:23:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 12:23:55.298486 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626211f-d2cc-4c64-80d6-77e92e7de0e7 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:55.299699 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626211f-d2cc-4c64-80d6-77e92e7de0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:55.300155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626211f-d2cc-4c64-80d6-77e92e7de0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:55.325263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-4626211f-d2cc-4c64-80d6-77e92e7de0e7 None None] MFA Rules not processed for user `803cf2e2fb664442ae8b0f8919a90aec`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:23:55.329791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4626211f-d2cc-4c64-80d6-77e92e7de0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:55.343638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1130/2289] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:23:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:23:58.705026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8684ef-f976-4a19-98a8-cae0367109bc None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:23:58.706387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8684ef-f976-4a19-98a8-cae0367109bc None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:23:58.706855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8684ef-f976-4a19-98a8-cae0367109bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:23:58.730637 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-4a8684ef-f976-4a19-98a8-cae0367109bc None None] MFA Rules not processed for user `07db808b777b4407ac05f3f0fa2394f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:23:58.735729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4a8684ef-f976-4a19-98a8-cae0367109bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:23:58.842141 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1160/2290] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:23:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:24:04.947881 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626211f-d2cc-4c64-80d6-77e92e7de0e7 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:24:04.947881 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4626211f-d2cc-4c64-80d6-77e92e7de0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:04.959043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626211f-d2cc-4c64-80d6-77e92e7de0e7 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:24:04.964772 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5adf370-8616-4d9c-b07e-eb725451225a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:24:04.972728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5adf370-8616-4d9c-b07e-eb725451225a service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:04.973236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5adf370-8616-4d9c-b07e-eb725451225a service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:04.973619 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5adf370-8616-4d9c-b07e-eb725451225a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:04.975746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-e5adf370-8616-4d9c-b07e-eb725451225a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:04.994239 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:24:05.049191 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5adf370-8616-4d9c-b07e-eb725451225a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:24:05.050769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5adf370-8616-4d9c-b07e-eb725451225a service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:24:05.063817 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1131/2291] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:24:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:24:05.664446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8684ef-f976-4a19-98a8-cae0367109bc None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:24:05.666708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4a8684ef-f976-4a19-98a8-cae0367109bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:05.687134 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8684ef-f976-4a19-98a8-cae0367109bc None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:24:05.694164 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709c05e-5b4b-4d5d-b19f-62362b9b5bc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:24:05.695239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709c05e-5b4b-4d5d-b19f-62362b9b5bc4 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:05.696307 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709c05e-5b4b-4d5d-b19f-62362b9b5bc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:05.696588 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709c05e-5b4b-4d5d-b19f-62362b9b5bc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:05.699219 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7709c05e-5b4b-4d5d-b19f-62362b9b5bc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:05.724939 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:24:05.775707 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709c05e-5b4b-4d5d-b19f-62362b9b5bc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:24:05.777470 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709c05e-5b4b-4d5d-b19f-62362b9b5bc4 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:24:05.788727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1161/2292] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:24:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:24:06.117472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5adf370-8616-4d9c-b07e-eb725451225a service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:24:06.130468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5adf370-8616-4d9c-b07e-eb725451225a service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:24:06.136098 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22497663-ccfd-4455-aa77-91b621684ecc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:24:06.136732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22497663-ccfd-4455-aa77-91b621684ecc service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:06.137020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22497663-ccfd-4455-aa77-91b621684ecc service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:06.137293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22497663-ccfd-4455-aa77-91b621684ecc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:06.139037 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-22497663-ccfd-4455-aa77-91b621684ecc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:06.206504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22497663-ccfd-4455-aa77-91b621684ecc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:24:06.208176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22497663-ccfd-4455-aa77-91b621684ecc service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:24:06.220989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1132/2293] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:24:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:24:06.926113 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709c05e-5b4b-4d5d-b19f-62362b9b5bc4 service nova] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:24:06.936181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709c05e-5b4b-4d5d-b19f-62362b9b5bc4 service nova] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:24:06.940552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730c326-4cdb-417f-b15b-709be1b617d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:24:06.944335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730c326-4cdb-417f-b15b-709be1b617d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:06.944335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730c326-4cdb-417f-b15b-709be1b617d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:06.944335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730c326-4cdb-417f-b15b-709be1b617d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:06.944335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b730c326-4cdb-417f-b15b-709be1b617d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:06.954584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730c326-4cdb-417f-b15b-709be1b617d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:24:06.956052 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730c326-4cdb-417f-b15b-709be1b617d2 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:24:06.964100 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1162/2294] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:24:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:24:07.656543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22497663-ccfd-4455-aa77-91b621684ecc service nova] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:24:07.658792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-22497663-ccfd-4455-aa77-91b621684ecc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:07.755783 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22497663-ccfd-4455-aa77-91b621684ecc service nova] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:24:07.761984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b974277-0c84-4e40-aaaf-281087743322 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ee8ba21bc184feeaafacab3e69e87bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:24:07.762603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b974277-0c84-4e40-aaaf-281087743322 service placement] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:07.763466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b974277-0c84-4e40-aaaf-281087743322 service placement] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:07.763904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b974277-0c84-4e40-aaaf-281087743322 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:07.765735 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9b974277-0c84-4e40-aaaf-281087743322 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:07.777997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b974277-0c84-4e40-aaaf-281087743322 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:24:07.779926 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b974277-0c84-4e40-aaaf-281087743322 service placement] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:24:07.791707 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1133/2295] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:24:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:24:14.801177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730c326-4cdb-417f-b15b-709be1b617d2 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:24:14.810568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730c326-4cdb-417f-b15b-709be1b617d2 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:24:14.815987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba73758-08c6-4835-8d8c-bd1e1edb552b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:24:14.816611 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba73758-08c6-4835-8d8c-bd1e1edb552b service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:14.816935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba73758-08c6-4835-8d8c-bd1e1edb552b service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:14.817206 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba73758-08c6-4835-8d8c-bd1e1edb552b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:14.819013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7ba73758-08c6-4835-8d8c-bd1e1edb552b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:14.829220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba73758-08c6-4835-8d8c-bd1e1edb552b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:24:14.830780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba73758-08c6-4835-8d8c-bd1e1edb552b service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:24:14.839919 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1163/2296] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:24:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:24:15.281956 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b974277-0c84-4e40-aaaf-281087743322 service placement] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:24:15.291234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b974277-0c84-4e40-aaaf-281087743322 service placement] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:24:15.295000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfffa2f-efc3-4773-a40c-01d79523786b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:24:15.295603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfffa2f-efc3-4773-a40c-01d79523786b service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:15.295887 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfffa2f-efc3-4773-a40c-01d79523786b service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:15.296159 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfffa2f-efc3-4773-a40c-01d79523786b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:15.298043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-5bfffa2f-efc3-4773-a40c-01d79523786b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:15.310514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfffa2f-efc3-4773-a40c-01d79523786b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:24:15.310514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfffa2f-efc3-4773-a40c-01d79523786b service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:24:15.318808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1134/2297] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:24:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:24:15.732421 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f9559-eacc-4268-87bc-1164126033f1 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:15.732747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f9559-eacc-4268-87bc-1164126033f1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:15.733532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f9559-eacc-4268-87bc-1164126033f1 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:15.734474 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1164/2298] 10.222.0.27 () {54 vars in 921 bytes} [Fri Aug 12 12:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:24:15.740833 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630715c8-518a-4365-bd06-f9faf8139f81 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:15.741126 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630715c8-518a-4365-bd06-f9faf8139f81 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:15.741431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630715c8-518a-4365-bd06-f9faf8139f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:15.761146 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-630715c8-518a-4365-bd06-f9faf8139f81 None None] MFA Rules not processed for user `687d0375abd44d2481f988c6df90fad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:24:15.762344 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-630715c8-518a-4365-bd06-f9faf8139f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:15.772351 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1135/2299] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Aug 12 12:24:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:24:15.784340 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f9559-eacc-4268-87bc-1164126033f1 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:24:15.786168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-cb1f9559-eacc-4268-87bc-1164126033f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:15.851846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f9559-eacc-4268-87bc-1164126033f1 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:24:15.856043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b08d3-2901-476b-9585-0f493c39b672 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe9f3fec7104a289e05e88e2d751458', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:24:15.856663 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b08d3-2901-476b-9585-0f493c39b672 service glance] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:15.856954 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b08d3-2901-476b-9585-0f493c39b672 service glance] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:15.857362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b08d3-2901-476b-9585-0f493c39b672 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:15.868628 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b08d3-2901-476b-9585-0f493c39b672 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:24:15.873589 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b08d3-2901-476b-9585-0f493c39b672 service glance] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:24:15.881296 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1165/2300] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:24:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:24:16.068184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d5233-f98a-4092-a923-2dc24e6645c3 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:16.068473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d5233-f98a-4092-a923-2dc24e6645c3 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:16.068774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d5233-f98a-4092-a923-2dc24e6645c3 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:16.070298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1136/2301] 10.222.0.27 () {54 vars in 921 bytes} [Fri Aug 12 12:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:24:16.075216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e9bad-7816-4bd4-b1a2-2d9e11cd0288 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:16.075504 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e9bad-7816-4bd4-b1a2-2d9e11cd0288 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:16.075781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e9bad-7816-4bd4-b1a2-2d9e11cd0288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:16.098363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-4b1e9bad-7816-4bd4-b1a2-2d9e11cd0288 None None] MFA Rules not processed for user `745fad9c109a4280906eb0349849117d`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:24:16.099465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4b1e9bad-7816-4bd4-b1a2-2d9e11cd0288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:16.106372 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1166/2302] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Aug 12 12:24:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:24:16.118866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d5233-f98a-4092-a923-2dc24e6645c3 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:24:16.131716 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d5233-f98a-4092-a923-2dc24e6645c3 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:24:16.135759 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af241329-649d-44a1-beec-5030f82240f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:24:16.136439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af241329-649d-44a1-beec-5030f82240f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:16.136917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af241329-649d-44a1-beec-5030f82240f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:16.137207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af241329-649d-44a1-beec-5030f82240f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:16.145375 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af241329-649d-44a1-beec-5030f82240f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:24:16.147049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af241329-649d-44a1-beec-5030f82240f7 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:24:16.155759 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1137/2303] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:24:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:24:16.240983 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274e5e9-36b7-4c90-9f91-e475a7e6d48f None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:16.241346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274e5e9-36b7-4c90-9f91-e475a7e6d48f None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:16.241753 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274e5e9-36b7-4c90-9f91-e475a7e6d48f None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:16.243127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1167/2304] 10.222.0.27 () {54 vars in 921 bytes} [Fri Aug 12 12:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:24:16.248669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dfb47-2769-4e67-a6d3-391e3518610b None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:16.248958 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dfb47-2769-4e67-a6d3-391e3518610b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:16.249266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dfb47-2769-4e67-a6d3-391e3518610b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:16.264188 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:24:16.270346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-f89dfb47-2769-4e67-a6d3-391e3518610b None None] MFA Rules not processed for user `687d0375abd44d2481f988c6df90fad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:24:16.271556 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f89dfb47-2769-4e67-a6d3-391e3518610b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:16.281076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1138/2305] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Aug 12 12:24:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:24:16.296094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274e5e9-36b7-4c90-9f91-e475a7e6d48f None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:24:16.303639 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274e5e9-36b7-4c90-9f91-e475a7e6d48f None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:24:16.308318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c7b86-b29f-4619-ac29-612369004ca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:24:16.313886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c7b86-b29f-4619-ac29-612369004ca2 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:16.313886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c7b86-b29f-4619-ac29-612369004ca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:16.313886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c7b86-b29f-4619-ac29-612369004ca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:16.321620 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c7b86-b29f-4619-ac29-612369004ca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:24:16.323060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c7b86-b29f-4619-ac29-612369004ca2 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:24:16.335052 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1168/2306] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:24:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:24:26.330686 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=64 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:24:26.514168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa67551-6e31-4c4e-9670-fba6197f1ecf None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:26.514675 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa67551-6e31-4c4e-9670-fba6197f1ecf None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:26.514771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa67551-6e31-4c4e-9670-fba6197f1ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:26.536333 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-1fa67551-6e31-4c4e-9670-fba6197f1ecf None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:24:26.537536 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1fa67551-6e31-4c4e-9670-fba6197f1ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:26.546159 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1139/2307] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Aug 12 12:24:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:24:26.563888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c7b86-b29f-4619-ac29-612369004ca2 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:24:26.575631 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c7b86-b29f-4619-ac29-612369004ca2 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:24:26.581936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67706e-0d96-41ce-a96e-ffa3f88a1513 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:24:26.582654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67706e-0d96-41ce-a96e-ffa3f88a1513 service swift] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:26.583071 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67706e-0d96-41ce-a96e-ffa3f88a1513 service swift] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:26.583395 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67706e-0d96-41ce-a96e-ffa3f88a1513 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:26.598369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67706e-0d96-41ce-a96e-ffa3f88a1513 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:24:26.603048 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67706e-0d96-41ce-a96e-ffa3f88a1513 service swift] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:24:26.611171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1169/2308] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:24:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 12:24:36.530097 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:24:36.607617 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=6 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:24:49.046866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa67551-6e31-4c4e-9670-fba6197f1ecf None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:24:49.063857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa67551-6e31-4c4e-9670-fba6197f1ecf None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:24:49.069726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43185224-7b72-4191-b0f0-30f7599cd7be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:24:49.070347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43185224-7b72-4191-b0f0-30f7599cd7be service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:24:49.070625 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43185224-7b72-4191-b0f0-30f7599cd7be service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:24:49.070905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43185224-7b72-4191-b0f0-30f7599cd7be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:24:49.072673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-43185224-7b72-4191-b0f0-30f7599cd7be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:24:49.086277 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43185224-7b72-4191-b0f0-30f7599cd7be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:24:49.088483 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43185224-7b72-4191-b0f0-30f7599cd7be service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:24:49.100767 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1140/2309] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:24:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:24:59.095711 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=6 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:25:03.976576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67706e-0d96-41ce-a96e-ffa3f88a1513 service swift] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:25:03.989591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67706e-0d96-41ce-a96e-ffa3f88a1513 service swift] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:25:03.995629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49228716-8637-4614-b148-fbd083eef43a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:25:03.996228 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49228716-8637-4614-b148-fbd083eef43a service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:03.996928 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49228716-8637-4614-b148-fbd083eef43a service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:03.996928 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49228716-8637-4614-b148-fbd083eef43a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:03.998699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-49228716-8637-4614-b148-fbd083eef43a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:25:04.011541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49228716-8637-4614-b148-fbd083eef43a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:25:04.013204 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49228716-8637-4614-b148-fbd083eef43a service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:25:04.024696 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1170/2310] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:25:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:25:13.620701 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a36c2-33eb-4952-8f75-8dcf31700a82 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:13.625537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a36c2-33eb-4952-8f75-8dcf31700a82 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:13.625537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a36c2-33eb-4952-8f75-8dcf31700a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:13.641907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-da7a36c2-33eb-4952-8f75-8dcf31700a82 None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:25:13.643129 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-da7a36c2-33eb-4952-8f75-8dcf31700a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:25:13.652043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1141/2311] 10.222.0.27 () {58 vars in 967 bytes} [Fri Aug 12 12:25:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:25:13.673794 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49228716-8637-4614-b148-fbd083eef43a service nova] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:25:13.686047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49228716-8637-4614-b148-fbd083eef43a service nova] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:25:13.691220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c2cb7-39ce-4e77-a81f-55f3c57a7bf2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:25:13.691817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c2cb7-39ce-4e77-a81f-55f3c57a7bf2 service swift] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:13.692102 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c2cb7-39ce-4e77-a81f-55f3c57a7bf2 service swift] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:13.692401 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c2cb7-39ce-4e77-a81f-55f3c57a7bf2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:13.701361 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c2cb7-39ce-4e77-a81f-55f3c57a7bf2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:25:13.705899 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c2cb7-39ce-4e77-a81f-55f3c57a7bf2 service swift] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:25:13.710915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1171/2312] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:25:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 12 12:25:22.998616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a36c2-33eb-4952-8f75-8dcf31700a82 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:25:23.009076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a36c2-33eb-4952-8f75-8dcf31700a82 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:25:23.012510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db556617-665c-49ae-80e1-d5794022d710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:25:23.013145 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db556617-665c-49ae-80e1-d5794022d710 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:23.013422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db556617-665c-49ae-80e1-d5794022d710 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:23.013694 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db556617-665c-49ae-80e1-d5794022d710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:23.022595 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db556617-665c-49ae-80e1-d5794022d710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:25:23.024013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db556617-665c-49ae-80e1-d5794022d710 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:25:23.033681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1142/2313] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:25:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:25:23.707750 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=12 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:25:24.985320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c2cb7-39ce-4e77-a81f-55f3c57a7bf2 service swift] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:25:24.996888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c2cb7-39ce-4e77-a81f-55f3c57a7bf2 service swift] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:25:25.002538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195609a4-86fb-4f19-8d3d-df065103684d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:25:25.003128 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195609a4-86fb-4f19-8d3d-df065103684d service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:25.003406 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195609a4-86fb-4f19-8d3d-df065103684d service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:25.003678 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195609a4-86fb-4f19-8d3d-df065103684d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:25.005447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-195609a4-86fb-4f19-8d3d-df065103684d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:25:25.017819 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195609a4-86fb-4f19-8d3d-df065103684d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:25:25.019616 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195609a4-86fb-4f19-8d3d-df065103684d service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:25:25.030221 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1172/2314] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:25:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:25:33.029351 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=9 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:25:35.025791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=6 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:25:56.922189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db556617-665c-49ae-80e1-d5794022d710 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:25:56.924127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-db556617-665c-49ae-80e1-d5794022d710 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:25:56.994783 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db556617-665c-49ae-80e1-d5794022d710 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:25:56.999259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25917829-bc3d-4e0e-a95a-fc716a1570b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:25:56.999886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25917829-bc3d-4e0e-a95a-fc716a1570b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:57.000172 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25917829-bc3d-4e0e-a95a-fc716a1570b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:57.000450 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25917829-bc3d-4e0e-a95a-fc716a1570b9 admin admin] PATH_INFO: `/v3/users/07db808b777b4407ac05f3f0fa2394f8` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:57.001610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25917829-bc3d-4e0e-a95a-fc716a1570b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07db808b777b4407ac05f3f0fa2394f8)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:25:57.003528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25917829-bc3d-4e0e-a95a-fc716a1570b9 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:25:57.137727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-25917829-bc3d-4e0e-a95a-fc716a1570b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07db808b777b4407ac05f3f0fa2394f8'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:25:57.138397 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-25917829-bc3d-4e0e-a95a-fc716a1570b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07db808b777b4407ac05f3f0fa2394f8'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:25:57.181849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-25917829-bc3d-4e0e-a95a-fc716a1570b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07db808b777b4407ac05f3f0fa2394f8'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:25:57.194839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-25917829-bc3d-4e0e-a95a-fc716a1570b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07db808b777b4407ac05f3f0fa2394f8'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:25:57.201949 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-25917829-bc3d-4e0e-a95a-fc716a1570b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07db808b777b4407ac05f3f0fa2394f8'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:25:57.207959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1143/2315] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:25:56 2022] DELETE /identity/v3/users/07db808b777b4407ac05f3f0fa2394f8 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:25:57.423817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195609a4-86fb-4f19-8d3d-df065103684d service nova] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:25:57.434947 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195609a4-86fb-4f19-8d3d-df065103684d service nova] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:25:57.493080 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec29902-5ffa-4715-8c59-474b530489bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:25:57.493707 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec29902-5ffa-4715-8c59-474b530489bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:57.493990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec29902-5ffa-4715-8c59-474b530489bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:57.494265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec29902-5ffa-4715-8c59-474b530489bd admin admin] PATH_INFO: `/v3/projects/f1b3e4986e72478bb46a86e1e9e3355c` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:57.495389 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec29902-5ffa-4715-8c59-474b530489bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b3e4986e72478bb46a86e1e9e3355c)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:25:57.497338 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec29902-5ffa-4715-8c59-474b530489bd admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:25:57.581712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5ec29902-5ffa-4715-8c59-474b530489bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b3e4986e72478bb46a86e1e9e3355c', 'request_id': 'req-5ec29902-5ffa-4715-8c59-474b530489bd'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:25:57.587693 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5ec29902-5ffa-4715-8c59-474b530489bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b3e4986e72478bb46a86e1e9e3355c', 'request_id': 'req-5ec29902-5ffa-4715-8c59-474b530489bd'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:25:57.594503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5ec29902-5ffa-4715-8c59-474b530489bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b3e4986e72478bb46a86e1e9e3355c', 'request_id': 'req-5ec29902-5ffa-4715-8c59-474b530489bd'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:25:57.679534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5ec29902-5ffa-4715-8c59-474b530489bd admin admin] The token cache is being invalidate because project f1b3e4986e72478bb46a86e1e9e3355c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:25:57.679920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5ec29902-5ffa-4715-8c59-474b530489bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:25:57.680600 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5ec29902-5ffa-4715-8c59-474b530489bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:25:57.683211 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1173/2316] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:25:57 2022] DELETE /identity/v3/projects/f1b3e4986e72478bb46a86e1e9e3355c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:25:57.688332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1f8c6-1c1a-4615-a980-1c5ba1d6801f None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:57.692777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1f8c6-1c1a-4615-a980-1c5ba1d6801f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:57.692777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1f8c6-1c1a-4615-a980-1c5ba1d6801f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:57.709424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-c8c1f8c6-1c1a-4615-a980-1c5ba1d6801f None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:25:57.710693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c8c1f8c6-1c1a-4615-a980-1c5ba1d6801f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:25:57.769105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1144/2317] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:25:57.777995 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc813d-2a14-4397-82f3-0da492ec683d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:57.778308 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc813d-2a14-4397-82f3-0da492ec683d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:57.778591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc813d-2a14-4397-82f3-0da492ec683d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:57.797766 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-1ebc813d-2a14-4397-82f3-0da492ec683d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:25:57.798982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-1ebc813d-2a14-4397-82f3-0da492ec683d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:25:57.807013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1174/2318] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:25:57.811186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1f8c6-1c1a-4615-a980-1c5ba1d6801f None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:25:57.821808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1f8c6-1c1a-4615-a980-1c5ba1d6801f None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:25:57.826377 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8ccc9-c77b-4288-b3b4-d077ef2530a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:25:57.826962 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8ccc9-c77b-4288-b3b4-d077ef2530a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:57.827241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8ccc9-c77b-4288-b3b4-d077ef2530a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:57.827512 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8ccc9-c77b-4288-b3b4-d077ef2530a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:57.828149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8ccc9-c77b-4288-b3b4-d077ef2530a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:25:57.828448 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8ccc9-c77b-4288-b3b4-d077ef2530a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:25:57.830062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8ccc9-c77b-4288-b3b4-d077ef2530a0 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:25:57.842216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1145/2319] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:25:57.846432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc813d-2a14-4397-82f3-0da492ec683d None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:25:57.855517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc813d-2a14-4397-82f3-0da492ec683d None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:25:57.860062 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9271df-6c18-42e3-8d9c-646ec8314dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:25:57.860658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9271df-6c18-42e3-8d9c-646ec8314dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:57.860956 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9271df-6c18-42e3-8d9c-646ec8314dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:57.861246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9271df-6c18-42e3-8d9c-646ec8314dc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:57.861855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9271df-6c18-42e3-8d9c-646ec8314dc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:25:57.863735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9271df-6c18-42e3-8d9c-646ec8314dc3 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:25:57.977464 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1175/2320] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:25:57 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:25:57.981577 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8ccc9-c77b-4288-b3b4-d077ef2530a0 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:25:57.991960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8ccc9-c77b-4288-b3b4-d077ef2530a0 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:25:57.996257 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:25:58.048972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a8363-56ee-431e-9a7a-c44ad9428cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:25:58.049598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a8363-56ee-431e-9a7a-c44ad9428cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:58.049883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a8363-56ee-431e-9a7a-c44ad9428cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:58.050159 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a8363-56ee-431e-9a7a-c44ad9428cb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:58.050786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a8363-56ee-431e-9a7a-c44ad9428cb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:25:58.052721 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a8363-56ee-431e-9a7a-c44ad9428cb1 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:25:58.132916 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1146/2321] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:25:57 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:25:58.134696 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9271df-6c18-42e3-8d9c-646ec8314dc3 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:25:58.145281 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9271df-6c18-42e3-8d9c-646ec8314dc3 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:25:58.150420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8846f2-b2b8-4e2d-9e6e-71490c1d5bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:25:58.153181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8846f2-b2b8-4e2d-9e6e-71490c1d5bfb admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:58.153181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8846f2-b2b8-4e2d-9e6e-71490c1d5bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:58.153181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8846f2-b2b8-4e2d-9e6e-71490c1d5bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:58.153181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8846f2-b2b8-4e2d-9e6e-71490c1d5bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:25:58.154862 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8846f2-b2b8-4e2d-9e6e-71490c1d5bfb admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:25:58.168103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1176/2322] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:25:58.169968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a8363-56ee-431e-9a7a-c44ad9428cb1 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:25:58.179525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a8363-56ee-431e-9a7a-c44ad9428cb1 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:25:58.185048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90d1fc-4d0f-4aec-a138-4f6381f59286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:25:58.185649 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90d1fc-4d0f-4aec-a138-4f6381f59286 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:58.185932 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90d1fc-4d0f-4aec-a138-4f6381f59286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:58.186235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90d1fc-4d0f-4aec-a138-4f6381f59286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:58.186884 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90d1fc-4d0f-4aec-a138-4f6381f59286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:25:58.188501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90d1fc-4d0f-4aec-a138-4f6381f59286 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:25:58.201058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1147/2323] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:25:58.205454 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8846f2-b2b8-4e2d-9e6e-71490c1d5bfb admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:25:58.215447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8846f2-b2b8-4e2d-9e6e-71490c1d5bfb admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:25:58.220045 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e130d83-cda9-407d-bcd9-8f234099b5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:25:58.220719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e130d83-cda9-407d-bcd9-8f234099b5a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:58.221002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e130d83-cda9-407d-bcd9-8f234099b5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:58.221269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e130d83-cda9-407d-bcd9-8f234099b5a8 admin admin] PATH_INFO: `/v3/projects/566d2428ccbe41888b9619faf5f212a2/users/a90e3591b4144fdfb8b5ed5391aec7c6/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:58.236963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e130d83-cda9-407d-bcd9-8f234099b5a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=566d2428ccbe41888b9619faf5f212a2, user_id=a90e3591b4144fdfb8b5ed5391aec7c6, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:25:58.239645 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e130d83-cda9-407d-bcd9-8f234099b5a8 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:25:58.308014 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1177/2324] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:25:58 2022] PUT /identity/v3/projects/566d2428ccbe41888b9619faf5f212a2/users/a90e3591b4144fdfb8b5ed5391aec7c6/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:25:58.313299 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37bb84e-92d1-4ff2-9416-3a4c5b338000 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:58.313658 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37bb84e-92d1-4ff2-9416-3a4c5b338000 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:58.313937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37bb84e-92d1-4ff2-9416-3a4c5b338000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:58.333729 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-c37bb84e-92d1-4ff2-9416-3a4c5b338000 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:25:58.335077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c37bb84e-92d1-4ff2-9416-3a4c5b338000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:25:58.392669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1148/2325] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:25:58.402467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e130d83-cda9-407d-bcd9-8f234099b5a8 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:25:58.404185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-8e130d83-cda9-407d-bcd9-8f234099b5a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:25:58.435026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:25:58.460350 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e130d83-cda9-407d-bcd9-8f234099b5a8 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:25:58.463983 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86a526-5a92-46a8-b493-4d34cf3893d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:25:58.464581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86a526-5a92-46a8-b493-4d34cf3893d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:25:58.464886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86a526-5a92-46a8-b493-4d34cf3893d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:25:58.465158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86a526-5a92-46a8-b493-4d34cf3893d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:25:58.474402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86a526-5a92-46a8-b493-4d34cf3893d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:25:58.475839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86a526-5a92-46a8-b493-4d34cf3893d0 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:25:58.485775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1178/2326] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:25:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:26:02.788225 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37bb84e-92d1-4ff2-9416-3a4c5b338000 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:26:02.788225 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c37bb84e-92d1-4ff2-9416-3a4c5b338000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:26:02.955004 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37bb84e-92d1-4ff2-9416-3a4c5b338000 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:26:02.961445 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b2548-cc96-47c4-8ea6-f5f5d1d5aa69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:26:02.962689 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b2548-cc96-47c4-8ea6-f5f5d1d5aa69 service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:26:02.963375 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b2548-cc96-47c4-8ea6-f5f5d1d5aa69 service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:26:02.963667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b2548-cc96-47c4-8ea6-f5f5d1d5aa69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:26:02.966620 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-ae2b2548-cc96-47c4-8ea6-f5f5d1d5aa69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:26:02.995486 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:26:03.057954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b2548-cc96-47c4-8ea6-f5f5d1d5aa69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:26:03.059986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b2548-cc96-47c4-8ea6-f5f5d1d5aa69 service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:26:03.073466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1149/2327] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:26:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:26:03.896130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bf4ad-0760-4dc2-817b-8f7eca6e2e57 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:26:03.896722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bf4ad-0760-4dc2-817b-8f7eca6e2e57 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:26:03.896722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bf4ad-0760-4dc2-817b-8f7eca6e2e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:26:03.935731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-315bf4ad-0760-4dc2-817b-8f7eca6e2e57 None None] MFA Rules not processed for user `a90e3591b4144fdfb8b5ed5391aec7c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:26:03.938137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-315bf4ad-0760-4dc2-817b-8f7eca6e2e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:26:04.057904 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1179/2328] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:26:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:26:04.058962 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b2548-cc96-47c4-8ea6-f5f5d1d5aa69 service nova] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:26:04.072375 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b2548-cc96-47c4-8ea6-f5f5d1d5aa69 service nova] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:26:04.077128 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a320423-dbca-45e7-9553-49ac936e2467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:26:04.077761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a320423-dbca-45e7-9553-49ac936e2467 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:26:04.078120 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a320423-dbca-45e7-9553-49ac936e2467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:26:04.078398 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a320423-dbca-45e7-9553-49ac936e2467 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:26:04.079058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a320423-dbca-45e7-9553-49ac936e2467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:26:04.081051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a320423-dbca-45e7-9553-49ac936e2467 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:26:04.178918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1150/2329] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:26:04 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:26:04.183171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bf4ad-0760-4dc2-817b-8f7eca6e2e57 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:26:04.194699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bf4ad-0760-4dc2-817b-8f7eca6e2e57 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:26:04.253959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b7f3b-7ad7-44c1-8fc8-f82c21566b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:26:04.254362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b7f3b-7ad7-44c1-8fc8-f82c21566b27 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:26:04.254643 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b7f3b-7ad7-44c1-8fc8-f82c21566b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:26:04.254915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b7f3b-7ad7-44c1-8fc8-f82c21566b27 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:26:04.255579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b7f3b-7ad7-44c1-8fc8-f82c21566b27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:26:04.257695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b7f3b-7ad7-44c1-8fc8-f82c21566b27 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:26:04.360568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1180/2330] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:26:04 2022] POST /identity/v3/users => generated 296 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:26:04.364882 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a320423-dbca-45e7-9553-49ac936e2467 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:26:04.375901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a320423-dbca-45e7-9553-49ac936e2467 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:26:04.381856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6565501-9c36-4e2c-89af-a467d94e209a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:26:04.382682 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6565501-9c36-4e2c-89af-a467d94e209a admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:26:04.383122 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6565501-9c36-4e2c-89af-a467d94e209a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:26:04.384084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6565501-9c36-4e2c-89af-a467d94e209a admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:26:04.384938 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6565501-9c36-4e2c-89af-a467d94e209a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:26:04.386892 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6565501-9c36-4e2c-89af-a467d94e209a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:26:04.403601 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1151/2331] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:26:04.408076 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b7f3b-7ad7-44c1-8fc8-f82c21566b27 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:26:04.421859 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b7f3b-7ad7-44c1-8fc8-f82c21566b27 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:26:04.426866 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce7b5b-2e9d-4db9-9264-cc571be6b5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:26:04.427633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce7b5b-2e9d-4db9-9264-cc571be6b5a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:26:04.427914 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce7b5b-2e9d-4db9-9264-cc571be6b5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:26:04.428223 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce7b5b-2e9d-4db9-9264-cc571be6b5a7 admin admin] PATH_INFO: `/v3/projects/82ceaffd799d43c2ad2fc4d6e3289cf9/users/9d2d4d28f99d4661b9cdfaa7832c4e2a/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:26:04.446859 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce7b5b-2e9d-4db9-9264-cc571be6b5a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ceaffd799d43c2ad2fc4d6e3289cf9, user_id=9d2d4d28f99d4661b9cdfaa7832c4e2a, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:26:04.450000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce7b5b-2e9d-4db9-9264-cc571be6b5a7 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:26:04.532922 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1181/2332] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:26:04 2022] PUT /identity/v3/projects/82ceaffd799d43c2ad2fc4d6e3289cf9/users/9d2d4d28f99d4661b9cdfaa7832c4e2a/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:26:04.536967 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6565501-9c36-4e2c-89af-a467d94e209a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:26:04.548660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6565501-9c36-4e2c-89af-a467d94e209a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:26:04.612022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1a4ed-a995-4ee3-9e22-5f9535d60ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:26:04.612670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1a4ed-a995-4ee3-9e22-5f9535d60ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:26:04.612947 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1a4ed-a995-4ee3-9e22-5f9535d60ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:26:04.613218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1a4ed-a995-4ee3-9e22-5f9535d60ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:26:04.613868 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1a4ed-a995-4ee3-9e22-5f9535d60ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:26:04.615596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1a4ed-a995-4ee3-9e22-5f9535d60ef3 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:26:04.627062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1152/2333] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:26:04.631416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce7b5b-2e9d-4db9-9264-cc571be6b5a7 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:26:04.644161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce7b5b-2e9d-4db9-9264-cc571be6b5a7 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:26:04.649176 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bf036-cfe5-4fcc-9712-c522a15209f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:26:04.650358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bf036-cfe5-4fcc-9712-c522a15209f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:26:04.650644 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bf036-cfe5-4fcc-9712-c522a15209f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:26:04.650906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bf036-cfe5-4fcc-9712-c522a15209f9 admin admin] PATH_INFO: `/v3/projects/82ceaffd799d43c2ad2fc4d6e3289cf9/users/9d2d4d28f99d4661b9cdfaa7832c4e2a/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:26:04.652917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bf036-cfe5-4fcc-9712-c522a15209f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ceaffd799d43c2ad2fc4d6e3289cf9, user_id=9d2d4d28f99d4661b9cdfaa7832c4e2a, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:26:04.655694 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bf036-cfe5-4fcc-9712-c522a15209f9 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:26:04.666948 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1182/2334] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:26:04 2022] PUT /identity/v3/projects/82ceaffd799d43c2ad2fc4d6e3289cf9/users/9d2d4d28f99d4661b9cdfaa7832c4e2a/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:26:10.158491 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a881561-0e13-4be4-85ac-c4ad8967e99e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:26:10.158491 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a881561-0e13-4be4-85ac-c4ad8967e99e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:26:10.158491 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a881561-0e13-4be4-85ac-c4ad8967e99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:26:10.186926 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-8a881561-0e13-4be4-85ac-c4ad8967e99e None None] MFA Rules not processed for user `9d2d4d28f99d4661b9cdfaa7832c4e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:26:10.186926 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8a881561-0e13-4be4-85ac-c4ad8967e99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:26:10.253020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1153/2335] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:26:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:26:10.266444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bf036-cfe5-4fcc-9712-c522a15209f9 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:26:10.268249 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-af5bf036-cfe5-4fcc-9712-c522a15209f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:26:10.284263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:26:10.349318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bf036-cfe5-4fcc-9712-c522a15209f9 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:26:10.355499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f2f71-1401-4280-9be8-03e08e579ed8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:26:10.356119 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f2f71-1401-4280-9be8-03e08e579ed8 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:26:10.356401 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f2f71-1401-4280-9be8-03e08e579ed8 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:26:10.356699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f2f71-1401-4280-9be8-03e08e579ed8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:26:10.413201 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f2f71-1401-4280-9be8-03e08e579ed8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:26:10.414787 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f2f71-1401-4280-9be8-03e08e579ed8 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:26:10.426035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1183/2336] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:26:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:26:20.250070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:26:20.420629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=51 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:26:38.206780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3dbd9b-0f54-49e2-bb6e-5bda24151e78 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:26:38.207795 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3dbd9b-0f54-49e2-bb6e-5bda24151e78 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:26:38.207906 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3dbd9b-0f54-49e2-bb6e-5bda24151e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:26:38.227214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-be3dbd9b-0f54-49e2-bb6e-5bda24151e78 None None] MFA Rules not processed for user `099b87a6c6b8409ab188c0b71ded7f55`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:26:38.228439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-be3dbd9b-0f54-49e2-bb6e-5bda24151e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:26:38.283233 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1154/2337] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:26:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:26:39.600129 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba71c1-711e-427d-9d03-834f6e7bb734 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:26:39.600610 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba71c1-711e-427d-9d03-834f6e7bb734 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:26:39.600902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba71c1-711e-427d-9d03-834f6e7bb734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:26:39.619711 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-7cba71c1-711e-427d-9d03-834f6e7bb734 None None] MFA Rules not processed for user `a90e3591b4144fdfb8b5ed5391aec7c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:26:39.620947 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7cba71c1-711e-427d-9d03-834f6e7bb734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:26:39.629016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1184/2338] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:26:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:26:48.276584 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=25 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:26:49.614332 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:26:55.383393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cc1bd-0fcc-4e4f-bf38-5fdeab75d984 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:26:55.383872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cc1bd-0fcc-4e4f-bf38-5fdeab75d984 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:26:55.384034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cc1bd-0fcc-4e4f-bf38-5fdeab75d984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:26:55.419179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-641cc1bd-0fcc-4e4f-bf38-5fdeab75d984 None None] MFA Rules not processed for user `0156c689e109489d9265dd3dd4d5cfde`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:26:55.420446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-641cc1bd-0fcc-4e4f-bf38-5fdeab75d984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:26:55.494546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1155/2339] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:26:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:26:59.074179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c25f80-b1f2-41f1-b406-75f23d3df9de None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:26:59.074687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c25f80-b1f2-41f1-b406-75f23d3df9de None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:26:59.074783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c25f80-b1f2-41f1-b406-75f23d3df9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:26:59.093873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-b7c25f80-b1f2-41f1-b406-75f23d3df9de None None] MFA Rules not processed for user `383cbf83186747a5b81a12ed5288c39a`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:26:59.096227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b7c25f80-b1f2-41f1-b406-75f23d3df9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:26:59.150707 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1185/2340] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:26:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:27:05.494805 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=37 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:27:09.144197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=25 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:27:09.257742 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641cc1bd-0fcc-4e4f-bf38-5fdeab75d984 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:27:09.340726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641cc1bd-0fcc-4e4f-bf38-5fdeab75d984 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:27:09.344299 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd09429-7d74-455a-bc7b-24ee61ae98d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:27:09.344933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd09429-7d74-455a-bc7b-24ee61ae98d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:27:09.345216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd09429-7d74-455a-bc7b-24ee61ae98d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:27:09.345547 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd09429-7d74-455a-bc7b-24ee61ae98d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:27:09.368207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd09429-7d74-455a-bc7b-24ee61ae98d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:27:09.369707 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd09429-7d74-455a-bc7b-24ee61ae98d1 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:27:09.379382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1156/2341] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:27:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:27:17.084394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c25f80-b1f2-41f1-b406-75f23d3df9de None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:27:17.114266 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c25f80-b1f2-41f1-b406-75f23d3df9de None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:27:17.119764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cb7f8-2394-4a40-8546-1b65b4fe8488 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:27:17.120354 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cb7f8-2394-4a40-8546-1b65b4fe8488 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:27:17.120656 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cb7f8-2394-4a40-8546-1b65b4fe8488 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:27:17.120938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cb7f8-2394-4a40-8546-1b65b4fe8488 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:27:17.122637 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2b3cb7f8-2394-4a40-8546-1b65b4fe8488 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:27:17.179621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cb7f8-2394-4a40-8546-1b65b4fe8488 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:27:17.181234 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cb7f8-2394-4a40-8546-1b65b4fe8488 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:27:17.190220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1186/2342] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:27:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:27:19.375566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=40 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:27:27.185963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=35 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:27:51.833570 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd09429-7d74-455a-bc7b-24ee61ae98d1 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:27:51.847345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd09429-7d74-455a-bc7b-24ee61ae98d1 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:27:51.852794 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd05e2e-64c9-4e37-beca-a787d2218797 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:27:51.853411 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd05e2e-64c9-4e37-beca-a787d2218797 service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:27:51.853691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd05e2e-64c9-4e37-beca-a787d2218797 service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:27:51.853964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd05e2e-64c9-4e37-beca-a787d2218797 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:27:51.855668 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-edd05e2e-64c9-4e37-beca-a787d2218797 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:27:51.915034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd05e2e-64c9-4e37-beca-a787d2218797 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:27:51.916619 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd05e2e-64c9-4e37-beca-a787d2218797 service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:27:51.925932 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1157/2343] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:27:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:27:59.622013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cb7f8-2394-4a40-8546-1b65b4fe8488 service nova] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:27:59.624120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2b3cb7f8-2394-4a40-8546-1b65b4fe8488 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:27:59.728382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cb7f8-2394-4a40-8546-1b65b4fe8488 service nova] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:27:59.733703 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf59ade-9d5c-432d-ad6d-13e19dfe7c30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ee8ba21bc184feeaafacab3e69e87bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:27:59.734393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf59ade-9d5c-432d-ad6d-13e19dfe7c30 service placement] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:27:59.734679 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf59ade-9d5c-432d-ad6d-13e19dfe7c30 service placement] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:27:59.735023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf59ade-9d5c-432d-ad6d-13e19dfe7c30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:27:59.736817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4bf59ade-9d5c-432d-ad6d-13e19dfe7c30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:27:59.750455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf59ade-9d5c-432d-ad6d-13e19dfe7c30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:27:59.752137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf59ade-9d5c-432d-ad6d-13e19dfe7c30 service placement] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:27:59.762087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1187/2344] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:27:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:01.921694 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=28 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:07.377855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268d246-c3da-49ae-8a04-07d14041df40 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:07.378516 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268d246-c3da-49ae-8a04-07d14041df40 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:07.378516 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268d246-c3da-49ae-8a04-07d14041df40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:07.407718 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-f268d246-c3da-49ae-8a04-07d14041df40 None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:07.408954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f268d246-c3da-49ae-8a04-07d14041df40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:07.466069 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1158/2345] 10.222.0.27 () {58 vars in 967 bytes} [Fri Aug 12 12:28:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:07.645092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf59ade-9d5c-432d-ad6d-13e19dfe7c30 service placement] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:07.646558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4bf59ade-9d5c-432d-ad6d-13e19dfe7c30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:07.714806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf59ade-9d5c-432d-ad6d-13e19dfe7c30 service placement] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:07.720585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b9094-82c8-49b8-ac08-64f585861717 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:07.721220 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b9094-82c8-49b8-ac08-64f585861717 service swift] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:07.721502 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b9094-82c8-49b8-ac08-64f585861717 service swift] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:07.721774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b9094-82c8-49b8-ac08-64f585861717 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:07.729653 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b9094-82c8-49b8-ac08-64f585861717 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:07.731250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b9094-82c8-49b8-ac08-64f585861717 service swift] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:07.740466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1188/2346] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:28:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 12:28:08.785448 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268d246-c3da-49ae-8a04-07d14041df40 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:08.786911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f268d246-c3da-49ae-8a04-07d14041df40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:08.812863 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268d246-c3da-49ae-8a04-07d14041df40 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:08.817321 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92aa188-0f7c-4ea5-a776-d6524e3c0545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:08.818057 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92aa188-0f7c-4ea5-a776-d6524e3c0545 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:08.818410 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92aa188-0f7c-4ea5-a776-d6524e3c0545 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:08.823373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92aa188-0f7c-4ea5-a776-d6524e3c0545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:08.825354 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f92aa188-0f7c-4ea5-a776-d6524e3c0545 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:08.912734 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92aa188-0f7c-4ea5-a776-d6524e3c0545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:08.914174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92aa188-0f7c-4ea5-a776-d6524e3c0545 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:08.933463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1159/2347] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:12.210445 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b9094-82c8-49b8-ac08-64f585861717 service swift] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:12.221666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b9094-82c8-49b8-ac08-64f585861717 service swift] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:12.225897 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14542a0-c1b2-4644-a207-19df10141ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:12.226602 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14542a0-c1b2-4644-a207-19df10141ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:12.226949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14542a0-c1b2-4644-a207-19df10141ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:12.227287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14542a0-c1b2-4644-a207-19df10141ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:12.229272 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a14542a0-c1b2-4644-a207-19df10141ea6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:12.243239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14542a0-c1b2-4644-a207-19df10141ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:12.244809 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14542a0-c1b2-4644-a207-19df10141ea6 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:12.255293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1189/2348] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:12.595291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a290b-8f19-4872-a348-c77de3939190 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:12.595820 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a290b-8f19-4872-a348-c77de3939190 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:12.595910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a290b-8f19-4872-a348-c77de3939190 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:12.597530 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1160/2349] 10.222.0.27 () {54 vars in 921 bytes} [Fri Aug 12 12:28:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:28:12.623761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fcfab-24a6-4cd6-b54b-94ad5bcc4ddb None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:12.624339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fcfab-24a6-4cd6-b54b-94ad5bcc4ddb None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:12.624616 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fcfab-24a6-4cd6-b54b-94ad5bcc4ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:12.678300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-a84fcfab-24a6-4cd6-b54b-94ad5bcc4ddb None None] MFA Rules not processed for user `5ee8ba21bc184feeaafacab3e69e87bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:12.679574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a84fcfab-24a6-4cd6-b54b-94ad5bcc4ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:12.689235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1190/2350] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Aug 12 12:28:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:12.701423 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a290b-8f19-4872-a348-c77de3939190 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:12.703766 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-434a290b-8f19-4872-a348-c77de3939190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:12.721736 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a290b-8f19-4872-a348-c77de3939190 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:12.727105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20c067-a528-4eb1-a915-d0d29f012015 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ee8ba21bc184feeaafacab3e69e87bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:12.727968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20c067-a528-4eb1-a915-d0d29f012015 service placement] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:12.728431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20c067-a528-4eb1-a915-d0d29f012015 service placement] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:12.728731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20c067-a528-4eb1-a915-d0d29f012015 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:12.738691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20c067-a528-4eb1-a915-d0d29f012015 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:12.740524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20c067-a528-4eb1-a915-d0d29f012015 service placement] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:12.751935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1161/2351] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:15.877805 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fcfab-24a6-4cd6-b54b-94ad5bcc4ddb None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:15.888554 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fcfab-24a6-4cd6-b54b-94ad5bcc4ddb None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:15.895582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc82497-f660-49b7-88bc-14d01c81b704 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ee8ba21bc184feeaafacab3e69e87bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:15.896197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc82497-f660-49b7-88bc-14d01c81b704 service placement] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:15.896476 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc82497-f660-49b7-88bc-14d01c81b704 service placement] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:15.896790 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc82497-f660-49b7-88bc-14d01c81b704 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:15.898568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-1bc82497-f660-49b7-88bc-14d01c81b704 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:15.911951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc82497-f660-49b7-88bc-14d01c81b704 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:15.913845 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc82497-f660-49b7-88bc-14d01c81b704 service placement] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:15.937119 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1191/2352] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:15.991144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20c067-a528-4eb1-a915-d0d29f012015 service placement] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:15.992763 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9f20c067-a528-4eb1-a915-d0d29f012015 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:16.007542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20c067-a528-4eb1-a915-d0d29f012015 service placement] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:16.012883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33d6e7-ad7c-4af4-a3b5-3c82d2a9039f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:16.013582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33d6e7-ad7c-4af4-a3b5-3c82d2a9039f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:16.014494 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33d6e7-ad7c-4af4-a3b5-3c82d2a9039f admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:16.014772 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33d6e7-ad7c-4af4-a3b5-3c82d2a9039f admin admin] PATH_INFO: `/v3/users/099b87a6c6b8409ab188c0b71ded7f55` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:16.016294 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33d6e7-ad7c-4af4-a3b5-3c82d2a9039f admin admin] RBAC: Authorizing `identity:delete_user(user_id=099b87a6c6b8409ab188c0b71ded7f55)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:16.017840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33d6e7-ad7c-4af4-a3b5-3c82d2a9039f admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:16.143923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:16.157393 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-9f33d6e7-ad7c-4af4-a3b5-3c82d2a9039f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '099b87a6c6b8409ab188c0b71ded7f55'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:16.158105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-9f33d6e7-ad7c-4af4-a3b5-3c82d2a9039f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '099b87a6c6b8409ab188c0b71ded7f55'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:16.213511 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-9f33d6e7-ad7c-4af4-a3b5-3c82d2a9039f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '099b87a6c6b8409ab188c0b71ded7f55'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:16.231533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-9f33d6e7-ad7c-4af4-a3b5-3c82d2a9039f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '099b87a6c6b8409ab188c0b71ded7f55'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:16.241042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-9f33d6e7-ad7c-4af4-a3b5-3c82d2a9039f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '099b87a6c6b8409ab188c0b71ded7f55'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:16.250572 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1162/2353] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:28:15 2022] DELETE /identity/v3/users/099b87a6c6b8409ab188c0b71ded7f55 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:16.531640 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc82497-f660-49b7-88bc-14d01c81b704 service placement] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:16.538418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:16.543121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc82497-f660-49b7-88bc-14d01c81b704 service placement] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:16.605887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f087008-15fc-45a5-a713-fff66a0ea2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:16.606497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f087008-15fc-45a5-a713-fff66a0ea2e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:16.606735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f087008-15fc-45a5-a713-fff66a0ea2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:16.606991 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f087008-15fc-45a5-a713-fff66a0ea2e6 admin admin] PATH_INFO: `/v3/projects/2e8def85acdd4e308bb51b4063c2ab4e` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:16.608740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f087008-15fc-45a5-a713-fff66a0ea2e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e8def85acdd4e308bb51b4063c2ab4e)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:16.610680 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f087008-15fc-45a5-a713-fff66a0ea2e6 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:16.730325 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5f087008-15fc-45a5-a713-fff66a0ea2e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e8def85acdd4e308bb51b4063c2ab4e', 'request_id': 'req-5f087008-15fc-45a5-a713-fff66a0ea2e6'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:16.740004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5f087008-15fc-45a5-a713-fff66a0ea2e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e8def85acdd4e308bb51b4063c2ab4e', 'request_id': 'req-5f087008-15fc-45a5-a713-fff66a0ea2e6'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:16.747339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5f087008-15fc-45a5-a713-fff66a0ea2e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e8def85acdd4e308bb51b4063c2ab4e', 'request_id': 'req-5f087008-15fc-45a5-a713-fff66a0ea2e6'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:16.815548 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5f087008-15fc-45a5-a713-fff66a0ea2e6 admin admin] The token cache is being invalidate because project 2e8def85acdd4e308bb51b4063c2ab4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:28:16.815847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5f087008-15fc-45a5-a713-fff66a0ea2e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:16.820978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5f087008-15fc-45a5-a713-fff66a0ea2e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:16.820978 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1192/2354] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:28:16 2022] DELETE /identity/v3/projects/2e8def85acdd4e308bb51b4063c2ab4e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:16.823872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4091c-871a-4592-ad07-2232377d5ca0 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:16.824032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4091c-871a-4592-ad07-2232377d5ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:16.824312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4091c-871a-4592-ad07-2232377d5ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:16.875643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-56f4091c-871a-4592-ad07-2232377d5ca0 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:16.876891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-56f4091c-871a-4592-ad07-2232377d5ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:16.937973 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1163/2355] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:16.947119 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccfe0d-3a23-4068-aa0f-c036cc8b28de None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:16.947950 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccfe0d-3a23-4068-aa0f-c036cc8b28de None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:16.948398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccfe0d-3a23-4068-aa0f-c036cc8b28de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:16.961623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4091c-871a-4592-ad07-2232377d5ca0 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:16.963355 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-56f4091c-871a-4592-ad07-2232377d5ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:16.976760 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4091c-871a-4592-ad07-2232377d5ca0 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:16.978530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-83ccfe0d-3a23-4068-aa0f-c036cc8b28de None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:16.979969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-83ccfe0d-3a23-4068-aa0f-c036cc8b28de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:16.982115 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f0f19-37bc-4f8a-8be3-1549ecb62ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:16.985240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f0f19-37bc-4f8a-8be3-1549ecb62ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:16.985240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f0f19-37bc-4f8a-8be3-1549ecb62ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:16.985240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f0f19-37bc-4f8a-8be3-1549ecb62ea2 admin admin] PATH_INFO: `/v3/users/fbd28b2571a540e7bdd9c15684dee82d` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:16.985240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f0f19-37bc-4f8a-8be3-1549ecb62ea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbd28b2571a540e7bdd9c15684dee82d)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:16.987357 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f0f19-37bc-4f8a-8be3-1549ecb62ea2 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:16.989615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1193/2356] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:16.993971 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccfe0d-3a23-4068-aa0f-c036cc8b28de None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:17.003494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccfe0d-3a23-4068-aa0f-c036cc8b28de None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:17.009045 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01d816-23d3-4666-bd39-f40c5c8d4a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:17.009713 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01d816-23d3-4666-bd39-f40c5c8d4a2f admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:17.010588 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01d816-23d3-4666-bd39-f40c5c8d4a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:17.010588 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01d816-23d3-4666-bd39-f40c5c8d4a2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:17.010883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01d816-23d3-4666-bd39-f40c5c8d4a2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:28:17.011479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01d816-23d3-4666-bd39-f40c5c8d4a2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:17.013304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01d816-23d3-4666-bd39-f40c5c8d4a2f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:17.026242 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1194/2357] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:28:17.030728 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01d816-23d3-4666-bd39-f40c5c8d4a2f admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:17.041116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01d816-23d3-4666-bd39-f40c5c8d4a2f admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:17.046429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a87f68-6662-4a1a-a8a3-88972321eb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:17.047024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a87f68-6662-4a1a-a8a3-88972321eb59 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:17.047306 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a87f68-6662-4a1a-a8a3-88972321eb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:17.047578 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a87f68-6662-4a1a-a8a3-88972321eb59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:17.048224 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a87f68-6662-4a1a-a8a3-88972321eb59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:17.050185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a87f68-6662-4a1a-a8a3-88972321eb59 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:17.123109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1195/2358] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:28:17 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:28:17.131567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a87f68-6662-4a1a-a8a3-88972321eb59 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:17.181136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1c8f0f19-37bc-4f8a-8be3-1549ecb62ea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbd28b2571a540e7bdd9c15684dee82d'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:17.181812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1c8f0f19-37bc-4f8a-8be3-1549ecb62ea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbd28b2571a540e7bdd9c15684dee82d'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:17.184889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:17.195311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a87f68-6662-4a1a-a8a3-88972321eb59 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:17.200077 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90132f8-0a86-4563-8047-3e52777925da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:17.200710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90132f8-0a86-4563-8047-3e52777925da admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:17.201004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90132f8-0a86-4563-8047-3e52777925da admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:17.201770 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90132f8-0a86-4563-8047-3e52777925da admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:17.202941 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90132f8-0a86-4563-8047-3e52777925da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:17.204894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90132f8-0a86-4563-8047-3e52777925da admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:17.235359 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1c8f0f19-37bc-4f8a-8be3-1549ecb62ea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbd28b2571a540e7bdd9c15684dee82d'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:17.248110 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1c8f0f19-37bc-4f8a-8be3-1549ecb62ea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbd28b2571a540e7bdd9c15684dee82d'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:17.255771 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1c8f0f19-37bc-4f8a-8be3-1549ecb62ea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbd28b2571a540e7bdd9c15684dee82d'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:17.259578 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:17.262415 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1164/2359] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:28:16 2022] DELETE /identity/v3/users/fbd28b2571a540e7bdd9c15684dee82d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:17.280203 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1196/2360] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:28:17 2022] POST /identity/v3/users => generated 286 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:28:17.282028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f0f19-37bc-4f8a-8be3-1549ecb62ea2 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:17.283712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1c8f0f19-37bc-4f8a-8be3-1549ecb62ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:17.355573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f0f19-37bc-4f8a-8be3-1549ecb62ea2 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:17.361421 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5e81b-816c-4c09-b2cc-7ef821e0ac72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:17.362032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5e81b-816c-4c09-b2cc-7ef821e0ac72 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:17.362314 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5e81b-816c-4c09-b2cc-7ef821e0ac72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:17.362860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5e81b-816c-4c09-b2cc-7ef821e0ac72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:17.363228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5e81b-816c-4c09-b2cc-7ef821e0ac72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:17.364876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5e81b-816c-4c09-b2cc-7ef821e0ac72 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:17.377584 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1165/2361] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:17.381714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90132f8-0a86-4563-8047-3e52777925da admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:17.393372 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90132f8-0a86-4563-8047-3e52777925da admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:17.398605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ff473-3fe1-4fe2-966d-93f43a709cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:17.399210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ff473-3fe1-4fe2-966d-93f43a709cbf admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:17.399494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ff473-3fe1-4fe2-966d-93f43a709cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:17.399987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ff473-3fe1-4fe2-966d-93f43a709cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:17.400577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ff473-3fe1-4fe2-966d-93f43a709cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:17.402207 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ff473-3fe1-4fe2-966d-93f43a709cbf admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:17.415655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1197/2362] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:17.417319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5e81b-816c-4c09-b2cc-7ef821e0ac72 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:17.428784 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5e81b-816c-4c09-b2cc-7ef821e0ac72 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:17.433624 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4faacd-4522-4fe3-84ca-21d6a3c908a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:17.434356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4faacd-4522-4fe3-84ca-21d6a3c908a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:17.436614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4faacd-4522-4fe3-84ca-21d6a3c908a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:17.440738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4faacd-4522-4fe3-84ca-21d6a3c908a8 admin admin] PATH_INFO: `/v3/projects/62f5a31606bb48e4a2977d6833f1186b/users/d3a9463dca854042a79e5b284e804477/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:17.451856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4faacd-4522-4fe3-84ca-21d6a3c908a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62f5a31606bb48e4a2977d6833f1186b, user_id=d3a9463dca854042a79e5b284e804477, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:17.454781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4faacd-4522-4fe3-84ca-21d6a3c908a8 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:17.519937 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ff473-3fe1-4fe2-966d-93f43a709cbf admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:17.519937 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-184ff473-3fe1-4fe2-966d-93f43a709cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:17.520265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1166/2363] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:28:17 2022] PUT /identity/v3/projects/62f5a31606bb48e4a2977d6833f1186b/users/d3a9463dca854042a79e5b284e804477/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:17.526562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771912f-6eeb-41be-983a-5658924ad27b None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:17.526961 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771912f-6eeb-41be-983a-5658924ad27b None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:17.527144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771912f-6eeb-41be-983a-5658924ad27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:17.534082 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ff473-3fe1-4fe2-966d-93f43a709cbf admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:17.564891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-8771912f-6eeb-41be-983a-5658924ad27b None None] MFA Rules not processed for user `d3a9463dca854042a79e5b284e804477`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:17.566659 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8771912f-6eeb-41be-983a-5658924ad27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:17.603446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec5642-1259-420f-9557-6fbf78734261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:17.604085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec5642-1259-420f-9557-6fbf78734261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:17.604370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec5642-1259-420f-9557-6fbf78734261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:17.604672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec5642-1259-420f-9557-6fbf78734261 admin admin] PATH_INFO: `/v3/projects/478411e9884545398aa7fbec4279a57b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:17.605817 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec5642-1259-420f-9557-6fbf78734261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=478411e9884545398aa7fbec4279a57b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:17.608113 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec5642-1259-420f-9557-6fbf78734261 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:17.639611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1167/2364] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:28:17.644202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771912f-6eeb-41be-983a-5658924ad27b None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:17.654955 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771912f-6eeb-41be-983a-5658924ad27b None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:17.660287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9d03a-9e9c-4603-8017-ff953d306b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:17.660920 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9d03a-9e9c-4603-8017-ff953d306b46 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:17.661332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9d03a-9e9c-4603-8017-ff953d306b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:17.661737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9d03a-9e9c-4603-8017-ff953d306b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:17.662449 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a9d03a-9e9c-4603-8017-ff953d306b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:17.664779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a9d03a-9e9c-4603-8017-ff953d306b46 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:17.715818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d2ec5642-1259-420f-9557-6fbf78734261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '478411e9884545398aa7fbec4279a57b', 'request_id': 'req-d2ec5642-1259-420f-9557-6fbf78734261'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:17.721758 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d2ec5642-1259-420f-9557-6fbf78734261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '478411e9884545398aa7fbec4279a57b', 'request_id': 'req-d2ec5642-1259-420f-9557-6fbf78734261'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:17.728450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d2ec5642-1259-420f-9557-6fbf78734261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '478411e9884545398aa7fbec4279a57b', 'request_id': 'req-d2ec5642-1259-420f-9557-6fbf78734261'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:17.751626 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1168/2365] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:28:17 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:28:17.755580 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9d03a-9e9c-4603-8017-ff953d306b46 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:17.795055 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d2ec5642-1259-420f-9557-6fbf78734261 admin admin] The token cache is being invalidate because project 478411e9884545398aa7fbec4279a57b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:28:17.795442 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d2ec5642-1259-420f-9557-6fbf78734261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:17.796836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d2ec5642-1259-420f-9557-6fbf78734261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:17.799430 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:17.801531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1198/2366] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:28:17 2022] DELETE /identity/v3/projects/478411e9884545398aa7fbec4279a57b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:17.802575 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec5642-1259-420f-9557-6fbf78734261 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:17.804027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-d2ec5642-1259-420f-9557-6fbf78734261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:17.819569 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9d03a-9e9c-4603-8017-ff953d306b46 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:17.824399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440abc90-d410-465c-b55f-aaa29d8f70e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:17.825047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440abc90-d410-465c-b55f-aaa29d8f70e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:17.825341 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440abc90-d410-465c-b55f-aaa29d8f70e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:17.825619 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440abc90-d410-465c-b55f-aaa29d8f70e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:17.826247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440abc90-d410-465c-b55f-aaa29d8f70e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:17.828958 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440abc90-d410-465c-b55f-aaa29d8f70e3 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:17.872732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec5642-1259-420f-9557-6fbf78734261 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:17.877624 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df20b7-8890-4a38-9c5d-e5b871e7d613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:17.878245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df20b7-8890-4a38-9c5d-e5b871e7d613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:17.878527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df20b7-8890-4a38-9c5d-e5b871e7d613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:17.878802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df20b7-8890-4a38-9c5d-e5b871e7d613 admin admin] PATH_INFO: `/v3/users/7bd00413b8d04ebbbe093cd16b63a25f` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:17.879982 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df20b7-8890-4a38-9c5d-e5b871e7d613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd00413b8d04ebbbe093cd16b63a25f)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:17.881930 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df20b7-8890-4a38-9c5d-e5b871e7d613 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:17.884177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:17.949994 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1169/2367] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:28:17 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:28:17.954978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440abc90-d410-465c-b55f-aaa29d8f70e3 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:17.956541 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-440abc90-d410-465c-b55f-aaa29d8f70e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:17.969666 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440abc90-d410-465c-b55f-aaa29d8f70e3 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:17.974437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7410e37-2755-4ede-9774-878de3bf34ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:17.975035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7410e37-2755-4ede-9774-878de3bf34ef admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:17.975335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7410e37-2755-4ede-9774-878de3bf34ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:17.975593 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7410e37-2755-4ede-9774-878de3bf34ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:17.976251 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7410e37-2755-4ede-9774-878de3bf34ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:17.977889 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7410e37-2755-4ede-9774-878de3bf34ef admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:17.989418 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1170/2368] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:28:17.993325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7410e37-2755-4ede-9774-878de3bf34ef admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:18.004691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7410e37-2755-4ede-9774-878de3bf34ef admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:18.009645 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd0c20-2f31-44a6-b866-bbe73b4caed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:18.011466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd0c20-2f31-44a6-b866-bbe73b4caed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:18.011466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd0c20-2f31-44a6-b866-bbe73b4caed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:18.011466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd0c20-2f31-44a6-b866-bbe73b4caed2 admin admin] PATH_INFO: `/v3/projects/19133cf2a5db4d2196fa5e6023cef0a0/users/a33525a9dd59488e9b416c30baca03eb/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:18.026417 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd0c20-2f31-44a6-b866-bbe73b4caed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19133cf2a5db4d2196fa5e6023cef0a0, user_id=a33525a9dd59488e9b416c30baca03eb, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:18.029148 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd0c20-2f31-44a6-b866-bbe73b4caed2 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:18.121237 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1171/2369] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:28:17 2022] PUT /identity/v3/projects/19133cf2a5db4d2196fa5e6023cef0a0/users/a33525a9dd59488e9b416c30baca03eb/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:28:18.125624 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd0c20-2f31-44a6-b866-bbe73b4caed2 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:18.129851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-47df20b7-8890-4a38-9c5d-e5b871e7d613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd00413b8d04ebbbe093cd16b63a25f'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:18.190902 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd0c20-2f31-44a6-b866-bbe73b4caed2 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:18.195579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56f4ae-95fa-49d1-97b7-52ab654c4087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:18.196186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56f4ae-95fa-49d1-97b7-52ab654c4087 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:18.196470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56f4ae-95fa-49d1-97b7-52ab654c4087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:18.196774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56f4ae-95fa-49d1-97b7-52ab654c4087 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:18.197446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56f4ae-95fa-49d1-97b7-52ab654c4087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:18.199030 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56f4ae-95fa-49d1-97b7-52ab654c4087 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:18.201240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-47df20b7-8890-4a38-9c5d-e5b871e7d613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd00413b8d04ebbbe093cd16b63a25f'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:18.202430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-47df20b7-8890-4a38-9c5d-e5b871e7d613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd00413b8d04ebbbe093cd16b63a25f'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:18.210201 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-47df20b7-8890-4a38-9c5d-e5b871e7d613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd00413b8d04ebbbe093cd16b63a25f'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:18.219204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1172/2370] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:18.219204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56f4ae-95fa-49d1-97b7-52ab654c4087 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:18.219204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6d56f4ae-95fa-49d1-97b7-52ab654c4087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:18.227038 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-47df20b7-8890-4a38-9c5d-e5b871e7d613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd00413b8d04ebbbe093cd16b63a25f'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:18.231389 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56f4ae-95fa-49d1-97b7-52ab654c4087 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:18.233152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1199/2371] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:28:17 2022] DELETE /identity/v3/users/7bd00413b8d04ebbbe093cd16b63a25f => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:28:18.294856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6a07f-c94f-4227-8a51-726d2aedaf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:18.295748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6a07f-c94f-4227-8a51-726d2aedaf83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:18.296097 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6a07f-c94f-4227-8a51-726d2aedaf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:18.296800 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6a07f-c94f-4227-8a51-726d2aedaf83 admin admin] PATH_INFO: `/v3/projects/19133cf2a5db4d2196fa5e6023cef0a0/users/a33525a9dd59488e9b416c30baca03eb/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:18.299477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6a07f-c94f-4227-8a51-726d2aedaf83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19133cf2a5db4d2196fa5e6023cef0a0, user_id=a33525a9dd59488e9b416c30baca03eb, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:18.302176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6a07f-c94f-4227-8a51-726d2aedaf83 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:18.315402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1173/2372] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:28:18 2022] PUT /identity/v3/projects/19133cf2a5db4d2196fa5e6023cef0a0/users/a33525a9dd59488e9b416c30baca03eb/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:28:18.331865 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b7d84-0c73-4eeb-98a7-5521d8243006 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:18.332154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b7d84-0c73-4eeb-98a7-5521d8243006 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:18.332431 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b7d84-0c73-4eeb-98a7-5521d8243006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:18.359945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-618b7d84-0c73-4eeb-98a7-5521d8243006 None None] MFA Rules not processed for user `a33525a9dd59488e9b416c30baca03eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:18.361275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-618b7d84-0c73-4eeb-98a7-5521d8243006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:18.432548 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1200/2373] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:18.444733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6a07f-c94f-4227-8a51-726d2aedaf83 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:18.446535 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-12b6a07f-c94f-4227-8a51-726d2aedaf83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:18.475797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:18.496123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b7d84-0c73-4eeb-98a7-5521d8243006 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:18.499092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-618b7d84-0c73-4eeb-98a7-5521d8243006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:18.547824 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6a07f-c94f-4227-8a51-726d2aedaf83 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:18.553149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38685519-a31f-4b59-8c09-982a8e633507 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:18.553539 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38685519-a31f-4b59-8c09-982a8e633507 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:18.553986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38685519-a31f-4b59-8c09-982a8e633507 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:18.554275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38685519-a31f-4b59-8c09-982a8e633507 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:18.566740 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38685519-a31f-4b59-8c09-982a8e633507 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:18.568183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38685519-a31f-4b59-8c09-982a8e633507 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:18.575740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b7d84-0c73-4eeb-98a7-5521d8243006 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:18.583204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1174/2374] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:18.583928 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de9abf-fea3-4e62-9bbc-a2ecc0ba5cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:18.586806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de9abf-fea3-4e62-9bbc-a2ecc0ba5cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:18.586806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de9abf-fea3-4e62-9bbc-a2ecc0ba5cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:18.586806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de9abf-fea3-4e62-9bbc-a2ecc0ba5cf4 admin admin] PATH_INFO: `/v3/projects/32c35b5151f54415a42230e5350c95dc` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:18.586806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de9abf-fea3-4e62-9bbc-a2ecc0ba5cf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32c35b5151f54415a42230e5350c95dc)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:18.588906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de9abf-fea3-4e62-9bbc-a2ecc0ba5cf4 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:18.597681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:18.716964 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-e7de9abf-fea3-4e62-9bbc-a2ecc0ba5cf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32c35b5151f54415a42230e5350c95dc', 'request_id': 'req-e7de9abf-fea3-4e62-9bbc-a2ecc0ba5cf4'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:18.722453 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-e7de9abf-fea3-4e62-9bbc-a2ecc0ba5cf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32c35b5151f54415a42230e5350c95dc', 'request_id': 'req-e7de9abf-fea3-4e62-9bbc-a2ecc0ba5cf4'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:18.728701 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-e7de9abf-fea3-4e62-9bbc-a2ecc0ba5cf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32c35b5151f54415a42230e5350c95dc', 'request_id': 'req-e7de9abf-fea3-4e62-9bbc-a2ecc0ba5cf4'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:18.810623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-e7de9abf-fea3-4e62-9bbc-a2ecc0ba5cf4 admin admin] The token cache is being invalidate because project 32c35b5151f54415a42230e5350c95dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:28:18.811562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-e7de9abf-fea3-4e62-9bbc-a2ecc0ba5cf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:18.812322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-e7de9abf-fea3-4e62-9bbc-a2ecc0ba5cf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:18.813886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1201/2375] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:28:18 2022] DELETE /identity/v3/projects/32c35b5151f54415a42230e5350c95dc => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:28:19.504673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38685519-a31f-4b59-8c09-982a8e633507 service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:19.516009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38685519-a31f-4b59-8c09-982a8e633507 service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:19.570853 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e368a77-8f50-43aa-b769-2d4a6bf0f540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:19.571480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e368a77-8f50-43aa-b769-2d4a6bf0f540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:19.571762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e368a77-8f50-43aa-b769-2d4a6bf0f540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:19.572033 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e368a77-8f50-43aa-b769-2d4a6bf0f540 admin admin] PATH_INFO: `/v3/users/d3a9463dca854042a79e5b284e804477` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:19.573178 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e368a77-8f50-43aa-b769-2d4a6bf0f540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3a9463dca854042a79e5b284e804477)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:19.575084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e368a77-8f50-43aa-b769-2d4a6bf0f540 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:19.763251 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-9e368a77-8f50-43aa-b769-2d4a6bf0f540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3a9463dca854042a79e5b284e804477'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:19.764450 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-9e368a77-8f50-43aa-b769-2d4a6bf0f540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3a9463dca854042a79e5b284e804477'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:19.824079 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-9e368a77-8f50-43aa-b769-2d4a6bf0f540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3a9463dca854042a79e5b284e804477'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:19.837633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-9e368a77-8f50-43aa-b769-2d4a6bf0f540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3a9463dca854042a79e5b284e804477'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:19.844528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-9e368a77-8f50-43aa-b769-2d4a6bf0f540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3a9463dca854042a79e5b284e804477'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:19.850528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1175/2376] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:28:19 2022] DELETE /identity/v3/users/d3a9463dca854042a79e5b284e804477 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:28:19.855108 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898856e-723f-43d1-a831-2d8c2714fdf4 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:19.855593 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898856e-723f-43d1-a831-2d8c2714fdf4 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:19.855695 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898856e-723f-43d1-a831-2d8c2714fdf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:19.874628 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-a898856e-723f-43d1-a831-2d8c2714fdf4 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:19.875833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a898856e-723f-43d1-a831-2d8c2714fdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:19.934383 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1202/2377] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:19.944782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e368a77-8f50-43aa-b769-2d4a6bf0f540 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:19.946171 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9e368a77-8f50-43aa-b769-2d4a6bf0f540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:19.977165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:20.003703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e368a77-8f50-43aa-b769-2d4a6bf0f540 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:20.007342 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e16d48-842c-45d7-9fa4-724aa8d442bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:20.007938 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e16d48-842c-45d7-9fa4-724aa8d442bd service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:20.011232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e16d48-842c-45d7-9fa4-724aa8d442bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:20.011232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e16d48-842c-45d7-9fa4-724aa8d442bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:20.018023 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e16d48-842c-45d7-9fa4-724aa8d442bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:20.019436 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e16d48-842c-45d7-9fa4-724aa8d442bd service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:20.028868 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1176/2378] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:20.428209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898856e-723f-43d1-a831-2d8c2714fdf4 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:20.428209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a898856e-723f-43d1-a831-2d8c2714fdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:20.441004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898856e-723f-43d1-a831-2d8c2714fdf4 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:20.445846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c310d53-226f-41db-a783-037065bd19c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:20.446485 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c310d53-226f-41db-a783-037065bd19c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:20.450613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c310d53-226f-41db-a783-037065bd19c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:20.450613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c310d53-226f-41db-a783-037065bd19c5 admin admin] PATH_INFO: `/v3/projects/62f5a31606bb48e4a2977d6833f1186b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:20.450613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c310d53-226f-41db-a783-037065bd19c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62f5a31606bb48e4a2977d6833f1186b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:20.450613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c310d53-226f-41db-a783-037065bd19c5 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:20.573970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5c310d53-226f-41db-a783-037065bd19c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62f5a31606bb48e4a2977d6833f1186b', 'request_id': 'req-5c310d53-226f-41db-a783-037065bd19c5'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:20.581029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5c310d53-226f-41db-a783-037065bd19c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62f5a31606bb48e4a2977d6833f1186b', 'request_id': 'req-5c310d53-226f-41db-a783-037065bd19c5'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:20.587995 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5c310d53-226f-41db-a783-037065bd19c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62f5a31606bb48e4a2977d6833f1186b', 'request_id': 'req-5c310d53-226f-41db-a783-037065bd19c5'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:20.686813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5c310d53-226f-41db-a783-037065bd19c5 admin admin] The token cache is being invalidate because project 62f5a31606bb48e4a2977d6833f1186b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:28:20.687036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5c310d53-226f-41db-a783-037065bd19c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:20.687716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5c310d53-226f-41db-a783-037065bd19c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:20.689227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1203/2379] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:28:20 2022] DELETE /identity/v3/projects/62f5a31606bb48e4a2977d6833f1186b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:20.693276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e16d48-842c-45d7-9fa4-724aa8d442bd service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:20.703113 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e16d48-842c-45d7-9fa4-724aa8d442bd service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:20.757977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debd45c-eeb5-4b8d-a80b-881dffd86f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:20.758629 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debd45c-eeb5-4b8d-a80b-881dffd86f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:20.758912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debd45c-eeb5-4b8d-a80b-881dffd86f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:20.759190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debd45c-eeb5-4b8d-a80b-881dffd86f4c admin admin] PATH_INFO: `/v3/users/a33525a9dd59488e9b416c30baca03eb` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:20.760451 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debd45c-eeb5-4b8d-a80b-881dffd86f4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a33525a9dd59488e9b416c30baca03eb)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:20.762401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debd45c-eeb5-4b8d-a80b-881dffd86f4c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:20.941520 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6debd45c-eeb5-4b8d-a80b-881dffd86f4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a33525a9dd59488e9b416c30baca03eb'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:20.942117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6debd45c-eeb5-4b8d-a80b-881dffd86f4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a33525a9dd59488e9b416c30baca03eb'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:21.000630 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6debd45c-eeb5-4b8d-a80b-881dffd86f4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a33525a9dd59488e9b416c30baca03eb'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:21.013091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6debd45c-eeb5-4b8d-a80b-881dffd86f4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a33525a9dd59488e9b416c30baca03eb'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:21.020322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6debd45c-eeb5-4b8d-a80b-881dffd86f4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a33525a9dd59488e9b416c30baca03eb'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:21.026254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1177/2380] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:28:20 2022] DELETE /identity/v3/users/a33525a9dd59488e9b416c30baca03eb => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:21.397760 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c310d53-226f-41db-a783-037065bd19c5 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:21.400192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-5c310d53-226f-41db-a783-037065bd19c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:21.416031 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c310d53-226f-41db-a783-037065bd19c5 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:21.434818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:21.477016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48091bdb-9ce8-4439-be76-6f9a5f07ee9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:21.477592 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48091bdb-9ce8-4439-be76-6f9a5f07ee9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:21.477876 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48091bdb-9ce8-4439-be76-6f9a5f07ee9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:21.478154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48091bdb-9ce8-4439-be76-6f9a5f07ee9e admin admin] PATH_INFO: `/v3/projects/19133cf2a5db4d2196fa5e6023cef0a0` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:21.479282 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48091bdb-9ce8-4439-be76-6f9a5f07ee9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=19133cf2a5db4d2196fa5e6023cef0a0)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:21.481235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48091bdb-9ce8-4439-be76-6f9a5f07ee9e admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:21.591057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-48091bdb-9ce8-4439-be76-6f9a5f07ee9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19133cf2a5db4d2196fa5e6023cef0a0', 'request_id': 'req-48091bdb-9ce8-4439-be76-6f9a5f07ee9e'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:21.597648 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-48091bdb-9ce8-4439-be76-6f9a5f07ee9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19133cf2a5db4d2196fa5e6023cef0a0', 'request_id': 'req-48091bdb-9ce8-4439-be76-6f9a5f07ee9e'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:21.604342 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-48091bdb-9ce8-4439-be76-6f9a5f07ee9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19133cf2a5db4d2196fa5e6023cef0a0', 'request_id': 'req-48091bdb-9ce8-4439-be76-6f9a5f07ee9e'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:21.692864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-48091bdb-9ce8-4439-be76-6f9a5f07ee9e admin admin] The token cache is being invalidate because project 19133cf2a5db4d2196fa5e6023cef0a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:28:21.699802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-48091bdb-9ce8-4439-be76-6f9a5f07ee9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:21.699802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-48091bdb-9ce8-4439-be76-6f9a5f07ee9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:21.699802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1204/2381] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:28:21 2022] DELETE /identity/v3/projects/19133cf2a5db4d2196fa5e6023cef0a0 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:21.700361 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2f052-ab41-4291-8f3d-a4789deae2eb None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:21.700849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2f052-ab41-4291-8f3d-a4789deae2eb None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:21.700944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2f052-ab41-4291-8f3d-a4789deae2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:21.722414 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-efb2f052-ab41-4291-8f3d-a4789deae2eb None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:21.723616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-efb2f052-ab41-4291-8f3d-a4789deae2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:21.773547 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:21.790651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1178/2382] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:21.802968 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a2ee6-9fa7-4d7d-8cab-12ba096ddf0a None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:21.803257 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a2ee6-9fa7-4d7d-8cab-12ba096ddf0a None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:21.806099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a2ee6-9fa7-4d7d-8cab-12ba096ddf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:21.841469 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9c5a2ee6-9fa7-4d7d-8cab-12ba096ddf0a None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:21.843338 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9c5a2ee6-9fa7-4d7d-8cab-12ba096ddf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:21.852393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1205/2383] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:21.856555 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2f052-ab41-4291-8f3d-a4789deae2eb None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:21.866830 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2f052-ab41-4291-8f3d-a4789deae2eb None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:21.871362 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5b32c-7af1-4712-9e64-c8b8980a96d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:21.871992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5b32c-7af1-4712-9e64-c8b8980a96d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:21.872277 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5b32c-7af1-4712-9e64-c8b8980a96d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:21.872548 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5b32c-7af1-4712-9e64-c8b8980a96d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:21.873255 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5b32c-7af1-4712-9e64-c8b8980a96d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:28:21.873560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5b32c-7af1-4712-9e64-c8b8980a96d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:21.875187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5b32c-7af1-4712-9e64-c8b8980a96d0 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:21.886954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1179/2384] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:28:21.892301 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a2ee6-9fa7-4d7d-8cab-12ba096ddf0a None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:21.901820 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a2ee6-9fa7-4d7d-8cab-12ba096ddf0a None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:21.907537 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce703c2-d9d0-41af-b3d9-ce326aa72500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:21.908144 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce703c2-d9d0-41af-b3d9-ce326aa72500 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:21.908425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce703c2-d9d0-41af-b3d9-ce326aa72500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:21.908729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce703c2-d9d0-41af-b3d9-ce326aa72500 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:21.909362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce703c2-d9d0-41af-b3d9-ce326aa72500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:21.911284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce703c2-d9d0-41af-b3d9-ce326aa72500 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:21.985277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1206/2385] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:28:21 2022] POST /identity/v3/projects => generated 381 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:28:21.989475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5b32c-7af1-4712-9e64-c8b8980a96d0 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:22.000187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5b32c-7af1-4712-9e64-c8b8980a96d0 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:22.057289 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a384a-563a-4d3d-a7ff-8633bd2aedce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:22.057805 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a384a-563a-4d3d-a7ff-8633bd2aedce admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:22.058087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a384a-563a-4d3d-a7ff-8633bd2aedce admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:22.058362 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a384a-563a-4d3d-a7ff-8633bd2aedce admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:22.058982 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a384a-563a-4d3d-a7ff-8633bd2aedce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:22.061579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a384a-563a-4d3d-a7ff-8633bd2aedce admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:22.146662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1180/2386] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:28:21 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:28:22.154870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce703c2-d9d0-41af-b3d9-ce326aa72500 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:22.165653 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce703c2-d9d0-41af-b3d9-ce326aa72500 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:22.170338 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7af26f-d5ee-4353-b730-0c63e487e197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:22.171037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7af26f-d5ee-4353-b730-0c63e487e197 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:22.171319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7af26f-d5ee-4353-b730-0c63e487e197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:22.171595 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7af26f-d5ee-4353-b730-0c63e487e197 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:22.172265 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7af26f-d5ee-4353-b730-0c63e487e197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:22.173899 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7af26f-d5ee-4353-b730-0c63e487e197 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:22.183973 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1207/2387] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:22.188699 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a384a-563a-4d3d-a7ff-8633bd2aedce admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:22.198722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a384a-563a-4d3d-a7ff-8633bd2aedce admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:22.204317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd5dcb-d9b3-4de1-adbf-2ad8b641b677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:22.204938 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd5dcb-d9b3-4de1-adbf-2ad8b641b677 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:22.205218 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd5dcb-d9b3-4de1-adbf-2ad8b641b677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:22.205488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd5dcb-d9b3-4de1-adbf-2ad8b641b677 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:22.206124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd5dcb-d9b3-4de1-adbf-2ad8b641b677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:22.207793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd5dcb-d9b3-4de1-adbf-2ad8b641b677 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:22.219569 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1181/2388] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:22.223945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7af26f-d5ee-4353-b730-0c63e487e197 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:22.234992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7af26f-d5ee-4353-b730-0c63e487e197 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:22.240061 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db91dcf-8373-4322-af37-98f758c4492d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:22.240744 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db91dcf-8373-4322-af37-98f758c4492d admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:22.241033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db91dcf-8373-4322-af37-98f758c4492d admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:22.241310 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db91dcf-8373-4322-af37-98f758c4492d admin admin] PATH_INFO: `/v3/projects/0ceb8abf9f524bf9a92627f363b9c9d9/users/bac3c29f097f4728a1e3902501bf7e89/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:22.256883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db91dcf-8373-4322-af37-98f758c4492d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ceb8abf9f524bf9a92627f363b9c9d9, user_id=bac3c29f097f4728a1e3902501bf7e89, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:22.259568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db91dcf-8373-4322-af37-98f758c4492d admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:22.362750 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1208/2389] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:28:22 2022] PUT /identity/v3/projects/0ceb8abf9f524bf9a92627f363b9c9d9/users/bac3c29f097f4728a1e3902501bf7e89/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:22.373069 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9d266-8bd9-4329-b437-3ca26322d2b4 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:22.374100 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9d266-8bd9-4329-b437-3ca26322d2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:22.374381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9d266-8bd9-4329-b437-3ca26322d2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:22.402438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-69f9d266-8bd9-4329-b437-3ca26322d2b4 None None] MFA Rules not processed for user `bac3c29f097f4728a1e3902501bf7e89`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:22.403649 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-69f9d266-8bd9-4329-b437-3ca26322d2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:22.467815 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1182/2390] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:22.472281 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db91dcf-8373-4322-af37-98f758c4492d admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:22.519615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:22.533285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db91dcf-8373-4322-af37-98f758c4492d admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:22.537925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44db76-3f04-4540-96fd-a6481586748b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:22.538558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44db76-3f04-4540-96fd-a6481586748b admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:22.538841 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44db76-3f04-4540-96fd-a6481586748b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:22.539117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44db76-3f04-4540-96fd-a6481586748b admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:22.539730 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44db76-3f04-4540-96fd-a6481586748b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:22.541670 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44db76-3f04-4540-96fd-a6481586748b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:22.643160 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1209/2391] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:28:22 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:28:22.647083 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9d266-8bd9-4329-b437-3ca26322d2b4 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:22.659118 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9d266-8bd9-4329-b437-3ca26322d2b4 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:22.699278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:22.714410 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31012d16-2106-4a9b-8c6e-4ae56c7ae368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:22.715020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31012d16-2106-4a9b-8c6e-4ae56c7ae368 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:22.715262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31012d16-2106-4a9b-8c6e-4ae56c7ae368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:22.715531 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31012d16-2106-4a9b-8c6e-4ae56c7ae368 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:22.716202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31012d16-2106-4a9b-8c6e-4ae56c7ae368 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:22.718759 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31012d16-2106-4a9b-8c6e-4ae56c7ae368 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:22.813379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1183/2392] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:28:22 2022] POST /identity/v3/users => generated 296 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:28:22.817490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44db76-3f04-4540-96fd-a6481586748b admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:22.828860 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44db76-3f04-4540-96fd-a6481586748b admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:22.835347 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604f108-ad64-452e-a1a8-528770f3619e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:22.835945 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604f108-ad64-452e-a1a8-528770f3619e admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:22.836241 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604f108-ad64-452e-a1a8-528770f3619e admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:22.836499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604f108-ad64-452e-a1a8-528770f3619e admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:22.837226 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604f108-ad64-452e-a1a8-528770f3619e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:22.838824 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604f108-ad64-452e-a1a8-528770f3619e admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:22.849295 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1210/2393] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:22.853427 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31012d16-2106-4a9b-8c6e-4ae56c7ae368 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:22.863232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31012d16-2106-4a9b-8c6e-4ae56c7ae368 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:22.867785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5460a29-40a6-4cc4-9c3e-550a3eb438a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:22.868456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5460a29-40a6-4cc4-9c3e-550a3eb438a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:22.868898 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5460a29-40a6-4cc4-9c3e-550a3eb438a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:22.869172 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5460a29-40a6-4cc4-9c3e-550a3eb438a6 admin admin] PATH_INFO: `/v3/projects/d7c3e188f0f24250857a3c4404ca8529/users/e9df2db192324b37a688f6a20c217520/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:22.885171 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5460a29-40a6-4cc4-9c3e-550a3eb438a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7c3e188f0f24250857a3c4404ca8529, user_id=e9df2db192324b37a688f6a20c217520, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:22.887928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5460a29-40a6-4cc4-9c3e-550a3eb438a6 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:22.939540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1184/2394] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:28:22 2022] PUT /identity/v3/projects/d7c3e188f0f24250857a3c4404ca8529/users/e9df2db192324b37a688f6a20c217520/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:28:22.943707 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604f108-ad64-452e-a1a8-528770f3619e admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:22.954488 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604f108-ad64-452e-a1a8-528770f3619e admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:23.012021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b271d2b-a4c8-40f8-9905-da1805596ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:23.012630 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b271d2b-a4c8-40f8-9905-da1805596ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:23.012939 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b271d2b-a4c8-40f8-9905-da1805596ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:23.013209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b271d2b-a4c8-40f8-9905-da1805596ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:23.013876 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b271d2b-a4c8-40f8-9905-da1805596ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:23.016330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b271d2b-a4c8-40f8-9905-da1805596ab5 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:23.026841 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1211/2395] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:23.031236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5460a29-40a6-4cc4-9c3e-550a3eb438a6 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:23.041922 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5460a29-40a6-4cc4-9c3e-550a3eb438a6 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:23.046761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40102f3-8701-4461-97e3-9e38e6c988b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:23.047409 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40102f3-8701-4461-97e3-9e38e6c988b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:23.047693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40102f3-8701-4461-97e3-9e38e6c988b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:23.048002 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40102f3-8701-4461-97e3-9e38e6c988b8 admin admin] PATH_INFO: `/v3/projects/d7c3e188f0f24250857a3c4404ca8529/users/e9df2db192324b37a688f6a20c217520/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:23.049955 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40102f3-8701-4461-97e3-9e38e6c988b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7c3e188f0f24250857a3c4404ca8529, user_id=e9df2db192324b37a688f6a20c217520, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:23.052684 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40102f3-8701-4461-97e3-9e38e6c988b8 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:23.067326 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1185/2396] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:28:23 2022] PUT /identity/v3/projects/d7c3e188f0f24250857a3c4404ca8529/users/e9df2db192324b37a688f6a20c217520/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:23.080112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bcfca-b6da-450d-9f22-0eeba84b7e4a None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:23.080502 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bcfca-b6da-450d-9f22-0eeba84b7e4a None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:23.080891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bcfca-b6da-450d-9f22-0eeba84b7e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:23.110000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f30bcfca-b6da-450d-9f22-0eeba84b7e4a None None] MFA Rules not processed for user `e9df2db192324b37a688f6a20c217520`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:23.111236 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f30bcfca-b6da-450d-9f22-0eeba84b7e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:23.178894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1212/2397] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:23.190215 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40102f3-8701-4461-97e3-9e38e6c988b8 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:23.191842 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b40102f3-8701-4461-97e3-9e38e6c988b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:23.250414 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40102f3-8701-4461-97e3-9e38e6c988b8 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:23.254070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073f0d5-b273-489b-b5a8-b1f776ccda35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:23.254696 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073f0d5-b273-489b-b5a8-b1f776ccda35 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:23.254979 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073f0d5-b273-489b-b5a8-b1f776ccda35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:23.255257 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073f0d5-b273-489b-b5a8-b1f776ccda35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:23.264828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073f0d5-b273-489b-b5a8-b1f776ccda35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:23.266249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073f0d5-b273-489b-b5a8-b1f776ccda35 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:23.275642 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1186/2398] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:23.485905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bfe05-d04b-4e5d-a5f2-211a834224d5 None None] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:23.486920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bfe05-d04b-4e5d-a5f2-211a834224d5 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:23.487727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731bfe05-d04b-4e5d-a5f2-211a834224d5 None None] PATH_INFO: `/` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:23.490180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1213/2399] 10.222.0.27 () {54 vars in 921 bytes} [Fri Aug 12 12:28:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:28:23.499084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec64e9-e2d1-41c2-9688-4e1446746a54 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:23.499084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec64e9-e2d1-41c2-9688-4e1446746a54 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:23.499084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec64e9-e2d1-41c2-9688-4e1446746a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:23.519807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-deec64e9-e2d1-41c2-9688-4e1446746a54 None None] MFA Rules not processed for user `745fad9c109a4280906eb0349849117d`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:23.521173 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-deec64e9-e2d1-41c2-9688-4e1446746a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:23.594253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1187/2400] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Aug 12 12:28:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:23.606441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731bfe05-d04b-4e5d-a5f2-211a834224d5 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:23.608467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-731bfe05-d04b-4e5d-a5f2-211a834224d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:23.622432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731bfe05-d04b-4e5d-a5f2-211a834224d5 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:23.628847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76541f-d357-4470-8866-c556b3def57c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:23.628847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76541f-d357-4470-8866-c556b3def57c service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:23.628847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76541f-d357-4470-8866-c556b3def57c service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:23.628847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76541f-d357-4470-8866-c556b3def57c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:23.637412 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76541f-d357-4470-8866-c556b3def57c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:23.639491 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76541f-d357-4470-8866-c556b3def57c service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:23.649255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1214/2401] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:23.736120 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17c7cb-09c5-4322-bec1-5fbad5219d01 None None] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:23.736592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17c7cb-09c5-4322-bec1-5fbad5219d01 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:23.736726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17c7cb-09c5-4322-bec1-5fbad5219d01 None None] PATH_INFO: `/` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:23.738278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1188/2402] 10.222.0.27 () {54 vars in 921 bytes} [Fri Aug 12 12:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 12 12:28:23.744364 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda01ee-2a31-435e-b69f-99160dab5168 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:23.744669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda01ee-2a31-435e-b69f-99160dab5168 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:23.744979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda01ee-2a31-435e-b69f-99160dab5168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:23.778654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-fdda01ee-2a31-435e-b69f-99160dab5168 None None] MFA Rules not processed for user `687d0375abd44d2481f988c6df90fad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:23.780452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-fdda01ee-2a31-435e-b69f-99160dab5168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:23.794153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:23.794730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17c7cb-09c5-4322-bec1-5fbad5219d01 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:23.860303 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:23.875474 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1215/2403] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Aug 12 12:28:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:23.877228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17c7cb-09c5-4322-bec1-5fbad5219d01 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:23.881954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc12398-6590-443e-b693-bc1fff62604c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:23.882587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc12398-6590-443e-b693-bc1fff62604c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:23.882873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc12398-6590-443e-b693-bc1fff62604c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:23.883152 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc12398-6590-443e-b693-bc1fff62604c admin admin] PATH_INFO: `/v3/users/bac3c29f097f4728a1e3902501bf7e89` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:23.884291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc12398-6590-443e-b693-bc1fff62604c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bac3c29f097f4728a1e3902501bf7e89)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:23.886303 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda01ee-2a31-435e-b69f-99160dab5168 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:23.886439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc12398-6590-443e-b693-bc1fff62604c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:23.896540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda01ee-2a31-435e-b69f-99160dab5168 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:23.900527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f793ae6-fdfe-4994-a808-98968c6a64c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:23.901213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f793ae6-fdfe-4994-a808-98968c6a64c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:23.901501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f793ae6-fdfe-4994-a808-98968c6a64c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:23.901779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f793ae6-fdfe-4994-a808-98968c6a64c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:23.913223 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f793ae6-fdfe-4994-a808-98968c6a64c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:23.914753 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f793ae6-fdfe-4994-a808-98968c6a64c9 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:23.927889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1216/2404] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:28:24.072900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-5fc12398-6590-443e-b693-bc1fff62604c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bac3c29f097f4728a1e3902501bf7e89'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:24.073620 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-5fc12398-6590-443e-b693-bc1fff62604c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bac3c29f097f4728a1e3902501bf7e89'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:24.144466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-5fc12398-6590-443e-b693-bc1fff62604c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bac3c29f097f4728a1e3902501bf7e89'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:24.156994 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-5fc12398-6590-443e-b693-bc1fff62604c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bac3c29f097f4728a1e3902501bf7e89'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:24.163593 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-5fc12398-6590-443e-b693-bc1fff62604c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bac3c29f097f4728a1e3902501bf7e89'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:24.169422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1189/2405] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:28:23 2022] DELETE /identity/v3/users/bac3c29f097f4728a1e3902501bf7e89 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:24.174406 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f75966-a8f6-4264-b13d-a69c450b4935 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:24.174791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f75966-a8f6-4264-b13d-a69c450b4935 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:24.175072 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f75966-a8f6-4264-b13d-a69c450b4935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:24.194330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-a0f75966-a8f6-4264-b13d-a69c450b4935 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:24.195537 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a0f75966-a8f6-4264-b13d-a69c450b4935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:24.255279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1217/2406] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:24.264872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc12398-6590-443e-b693-bc1fff62604c admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:24.266290 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-5fc12398-6590-443e-b693-bc1fff62604c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:24.325704 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc12398-6590-443e-b693-bc1fff62604c admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:24.329922 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6c971-7a58-4c09-a95f-55301e0a716e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:24.330568 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6c971-7a58-4c09-a95f-55301e0a716e service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:24.330857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6c971-7a58-4c09-a95f-55301e0a716e service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:24.331132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6c971-7a58-4c09-a95f-55301e0a716e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:24.340724 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6c971-7a58-4c09-a95f-55301e0a716e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:24.342917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6c971-7a58-4c09-a95f-55301e0a716e service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:24.352932 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1190/2407] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:24.837831 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f75966-a8f6-4264-b13d-a69c450b4935 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:24.839267 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a0f75966-a8f6-4264-b13d-a69c450b4935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:24.851705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f75966-a8f6-4264-b13d-a69c450b4935 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:24.856855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b83026-2557-480f-a69d-d76a20961d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:24.858906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b83026-2557-480f-a69d-d76a20961d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:24.858906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b83026-2557-480f-a69d-d76a20961d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:24.858906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b83026-2557-480f-a69d-d76a20961d15 admin admin] PATH_INFO: `/v3/projects/0ceb8abf9f524bf9a92627f363b9c9d9` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:24.860455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b83026-2557-480f-a69d-d76a20961d15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ceb8abf9f524bf9a92627f363b9c9d9)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:24.862433 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b83026-2557-480f-a69d-d76a20961d15 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:24.997204 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c4b83026-2557-480f-a69d-d76a20961d15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ceb8abf9f524bf9a92627f363b9c9d9', 'request_id': 'req-c4b83026-2557-480f-a69d-d76a20961d15'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:25.002955 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c4b83026-2557-480f-a69d-d76a20961d15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ceb8abf9f524bf9a92627f363b9c9d9', 'request_id': 'req-c4b83026-2557-480f-a69d-d76a20961d15'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:25.009293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c4b83026-2557-480f-a69d-d76a20961d15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ceb8abf9f524bf9a92627f363b9c9d9', 'request_id': 'req-c4b83026-2557-480f-a69d-d76a20961d15'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:25.091690 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c4b83026-2557-480f-a69d-d76a20961d15 admin admin] The token cache is being invalidate because project 0ceb8abf9f524bf9a92627f363b9c9d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:28:25.092688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c4b83026-2557-480f-a69d-d76a20961d15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:25.093325 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c4b83026-2557-480f-a69d-d76a20961d15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:25.094784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1218/2408] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:28:24 2022] DELETE /identity/v3/projects/0ceb8abf9f524bf9a92627f363b9c9d9 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:25.098948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6c971-7a58-4c09-a95f-55301e0a716e service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:25.109862 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6c971-7a58-4c09-a95f-55301e0a716e service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:25.166972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754be36f-fc14-4bfa-af53-f2a0882ca81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:25.167588 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754be36f-fc14-4bfa-af53-f2a0882ca81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:25.167880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754be36f-fc14-4bfa-af53-f2a0882ca81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:25.168137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754be36f-fc14-4bfa-af53-f2a0882ca81e admin admin] PATH_INFO: `/v3/users/e9df2db192324b37a688f6a20c217520` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:25.169236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754be36f-fc14-4bfa-af53-f2a0882ca81e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9df2db192324b37a688f6a20c217520)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:25.171780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754be36f-fc14-4bfa-af53-f2a0882ca81e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:25.184084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:25.288981 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-754be36f-fc14-4bfa-af53-f2a0882ca81e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9df2db192324b37a688f6a20c217520'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:25.289648 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-754be36f-fc14-4bfa-af53-f2a0882ca81e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9df2db192324b37a688f6a20c217520'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:25.349940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-754be36f-fc14-4bfa-af53-f2a0882ca81e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9df2db192324b37a688f6a20c217520'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:25.362697 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-754be36f-fc14-4bfa-af53-f2a0882ca81e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9df2db192324b37a688f6a20c217520'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:25.369512 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-754be36f-fc14-4bfa-af53-f2a0882ca81e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9df2db192324b37a688f6a20c217520'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:25.375693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1191/2409] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:28:25 2022] DELETE /identity/v3/users/e9df2db192324b37a688f6a20c217520 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:25.803812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b83026-2557-480f-a69d-d76a20961d15 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:25.805884 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-c4b83026-2557-480f-a69d-d76a20961d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:25.823658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b83026-2557-480f-a69d-d76a20961d15 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:25.844266 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:25.887783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f458095-bfdc-485f-a89e-fd1492f74da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:25.888454 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f458095-bfdc-485f-a89e-fd1492f74da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:25.888772 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f458095-bfdc-485f-a89e-fd1492f74da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:25.890317 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f458095-bfdc-485f-a89e-fd1492f74da7 admin admin] PATH_INFO: `/v3/projects/d7c3e188f0f24250857a3c4404ca8529` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:25.891485 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f458095-bfdc-485f-a89e-fd1492f74da7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7c3e188f0f24250857a3c4404ca8529)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:25.893512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f458095-bfdc-485f-a89e-fd1492f74da7 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:25.990927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1f458095-bfdc-485f-a89e-fd1492f74da7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7c3e188f0f24250857a3c4404ca8529', 'request_id': 'req-1f458095-bfdc-485f-a89e-fd1492f74da7'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:25.996581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1f458095-bfdc-485f-a89e-fd1492f74da7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7c3e188f0f24250857a3c4404ca8529', 'request_id': 'req-1f458095-bfdc-485f-a89e-fd1492f74da7'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:26.004345 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1f458095-bfdc-485f-a89e-fd1492f74da7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7c3e188f0f24250857a3c4404ca8529', 'request_id': 'req-1f458095-bfdc-485f-a89e-fd1492f74da7'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:26.064319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1f458095-bfdc-485f-a89e-fd1492f74da7 admin admin] The token cache is being invalidate because project d7c3e188f0f24250857a3c4404ca8529 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:28:26.064844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1f458095-bfdc-485f-a89e-fd1492f74da7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:26.066048 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1f458095-bfdc-485f-a89e-fd1492f74da7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:26.067699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1219/2410] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:28:25 2022] DELETE /identity/v3/projects/d7c3e188f0f24250857a3c4404ca8529 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:26.072525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fed3f7-16d5-42c4-9538-2e41b36849b5 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:26.073240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fed3f7-16d5-42c4-9538-2e41b36849b5 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:26.073240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fed3f7-16d5-42c4-9538-2e41b36849b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:26.096363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-89fed3f7-16d5-42c4-9538-2e41b36849b5 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:26.097620 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-89fed3f7-16d5-42c4-9538-2e41b36849b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:26.157923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1192/2411] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:26.165729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285ae34-4eaa-4895-8039-e674fb2a7e4e None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:26.166016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285ae34-4eaa-4895-8039-e674fb2a7e4e None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:26.166317 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285ae34-4eaa-4895-8039-e674fb2a7e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:26.187180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-4285ae34-4eaa-4895-8039-e674fb2a7e4e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:26.188384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4285ae34-4eaa-4895-8039-e674fb2a7e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:26.196191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1220/2412] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:26.200495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fed3f7-16d5-42c4-9538-2e41b36849b5 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:26.210726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fed3f7-16d5-42c4-9538-2e41b36849b5 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:26.215396 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04b0ee-3108-4911-b832-d622fc6e31a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:26.216011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04b0ee-3108-4911-b832-d622fc6e31a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:26.216298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04b0ee-3108-4911-b832-d622fc6e31a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:26.216574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04b0ee-3108-4911-b832-d622fc6e31a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:26.217265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04b0ee-3108-4911-b832-d622fc6e31a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:28:26.217696 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04b0ee-3108-4911-b832-d622fc6e31a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:26.219298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04b0ee-3108-4911-b832-d622fc6e31a9 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:26.231058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1193/2413] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:28:26.235995 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285ae34-4eaa-4895-8039-e674fb2a7e4e None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:26.245562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285ae34-4eaa-4895-8039-e674fb2a7e4e None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:26.251047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506d475-96c0-41b8-b94d-4c61c5a02991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:26.251651 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506d475-96c0-41b8-b94d-4c61c5a02991 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:26.251934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506d475-96c0-41b8-b94d-4c61c5a02991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:26.252210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506d475-96c0-41b8-b94d-4c61c5a02991 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:26.252878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506d475-96c0-41b8-b94d-4c61c5a02991 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:26.254801 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506d475-96c0-41b8-b94d-4c61c5a02991 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:26.369585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1221/2414] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:28:26 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:28:26.373906 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04b0ee-3108-4911-b832-d622fc6e31a9 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:26.384171 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04b0ee-3108-4911-b832-d622fc6e31a9 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:26.440487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab768e-d8bf-4976-8aa6-734c2f7f04d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:26.441031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab768e-d8bf-4976-8aa6-734c2f7f04d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:26.441312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab768e-d8bf-4976-8aa6-734c2f7f04d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:26.441582 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab768e-d8bf-4976-8aa6-734c2f7f04d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:26.442196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab768e-d8bf-4976-8aa6-734c2f7f04d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:26.444401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab768e-d8bf-4976-8aa6-734c2f7f04d1 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:26.527544 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1194/2415] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:28:26 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:28:26.532358 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506d475-96c0-41b8-b94d-4c61c5a02991 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:26.542757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506d475-96c0-41b8-b94d-4c61c5a02991 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:26.548038 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157cbc0-a41a-4151-8bbd-c01c7762cfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:26.548629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157cbc0-a41a-4151-8bbd-c01c7762cfbc admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:26.554435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157cbc0-a41a-4151-8bbd-c01c7762cfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:26.554435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157cbc0-a41a-4151-8bbd-c01c7762cfbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:26.554435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157cbc0-a41a-4151-8bbd-c01c7762cfbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:26.554435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157cbc0-a41a-4151-8bbd-c01c7762cfbc admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:26.561921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1222/2416] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:26.565803 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab768e-d8bf-4976-8aa6-734c2f7f04d1 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:26.575591 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab768e-d8bf-4976-8aa6-734c2f7f04d1 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:26.580423 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60249f24-969e-4f84-9f2c-d796f6991c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:26.581037 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60249f24-969e-4f84-9f2c-d796f6991c9d admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:26.581314 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60249f24-969e-4f84-9f2c-d796f6991c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:26.581584 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60249f24-969e-4f84-9f2c-d796f6991c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:26.582220 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60249f24-969e-4f84-9f2c-d796f6991c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:26.583827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60249f24-969e-4f84-9f2c-d796f6991c9d admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:26.585076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:26.595725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1195/2417] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:26.599770 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157cbc0-a41a-4151-8bbd-c01c7762cfbc admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:26.610342 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157cbc0-a41a-4151-8bbd-c01c7762cfbc admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:26.614896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afb065-8427-4e00-8587-342456e5660b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:26.615561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afb065-8427-4e00-8587-342456e5660b admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:26.615842 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afb065-8427-4e00-8587-342456e5660b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:26.616110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afb065-8427-4e00-8587-342456e5660b admin admin] PATH_INFO: `/v3/projects/944825a0f0ee48beab1f0bb7cbb39859/users/588f7a764c224d9a88baa3a3ed8de864/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:26.631160 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afb065-8427-4e00-8587-342456e5660b admin admin] RBAC: Authorizing `identity:create_grant(project_id=944825a0f0ee48beab1f0bb7cbb39859, user_id=588f7a764c224d9a88baa3a3ed8de864, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:26.637081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afb065-8427-4e00-8587-342456e5660b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:26.682244 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1223/2418] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:28:26 2022] PUT /identity/v3/projects/944825a0f0ee48beab1f0bb7cbb39859/users/588f7a764c224d9a88baa3a3ed8de864/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:26.693118 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad0639-1e58-406c-b6d9-9a248fb0ef5e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:26.693338 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad0639-1e58-406c-b6d9-9a248fb0ef5e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:26.693623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad0639-1e58-406c-b6d9-9a248fb0ef5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:26.722058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-adad0639-1e58-406c-b6d9-9a248fb0ef5e None None] MFA Rules not processed for user `588f7a764c224d9a88baa3a3ed8de864`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:26.723308 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-adad0639-1e58-406c-b6d9-9a248fb0ef5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:26.787667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1196/2419] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:26.797894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afb065-8427-4e00-8587-342456e5660b admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:26.855894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:26.871437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afb065-8427-4e00-8587-342456e5660b admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:26.877449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99169626-9cb8-4b22-a5f9-e280c58f57c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:26.878081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99169626-9cb8-4b22-a5f9-e280c58f57c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:26.878394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99169626-9cb8-4b22-a5f9-e280c58f57c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:26.878629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99169626-9cb8-4b22-a5f9-e280c58f57c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:26.879274 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99169626-9cb8-4b22-a5f9-e280c58f57c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:26.881384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99169626-9cb8-4b22-a5f9-e280c58f57c5 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:26.993668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1224/2420] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:28:26 2022] POST /identity/v3/projects => generated 381 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:28:26.997701 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad0639-1e58-406c-b6d9-9a248fb0ef5e None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:27.010399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad0639-1e58-406c-b6d9-9a248fb0ef5e None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:27.074396 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d8c87-c4eb-435c-bec4-ebfe055b841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:27.075054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d8c87-c4eb-435c-bec4-ebfe055b841d admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:27.075353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d8c87-c4eb-435c-bec4-ebfe055b841d admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:27.075627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d8c87-c4eb-435c-bec4-ebfe055b841d admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:27.076253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d8c87-c4eb-435c-bec4-ebfe055b841d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:27.078492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d8c87-c4eb-435c-bec4-ebfe055b841d admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:27.162725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1197/2421] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:28:26 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:28:27.166672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99169626-9cb8-4b22-a5f9-e280c58f57c5 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:27.177377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99169626-9cb8-4b22-a5f9-e280c58f57c5 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:27.183229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b6339-20a3-4086-ada8-1ea7a34cad20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:27.183826 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b6339-20a3-4086-ada8-1ea7a34cad20 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:27.184108 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b6339-20a3-4086-ada8-1ea7a34cad20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:27.184379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b6339-20a3-4086-ada8-1ea7a34cad20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:27.185158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b6339-20a3-4086-ada8-1ea7a34cad20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:27.187720 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b6339-20a3-4086-ada8-1ea7a34cad20 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:27.201101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1225/2422] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:27.202984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d8c87-c4eb-435c-bec4-ebfe055b841d admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:27.213513 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d8c87-c4eb-435c-bec4-ebfe055b841d admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:27.219057 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6323d-a51e-46b6-915f-1383a4eac35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:27.219709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6323d-a51e-46b6-915f-1383a4eac35b admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:27.219992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6323d-a51e-46b6-915f-1383a4eac35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:27.220270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6323d-a51e-46b6-915f-1383a4eac35b admin admin] PATH_INFO: `/v3/projects/d1bb06502cac4db983a29bdf8c9c6a0a/users/6e6a8a4cf4094b77826d6ee116efcc64/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:27.236008 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6323d-a51e-46b6-915f-1383a4eac35b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1bb06502cac4db983a29bdf8c9c6a0a, user_id=6e6a8a4cf4094b77826d6ee116efcc64, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:27.238626 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6323d-a51e-46b6-915f-1383a4eac35b admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:27.305537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1198/2423] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:28:27 2022] PUT /identity/v3/projects/d1bb06502cac4db983a29bdf8c9c6a0a/users/6e6a8a4cf4094b77826d6ee116efcc64/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:27.309552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b6339-20a3-4086-ada8-1ea7a34cad20 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:27.319975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b6339-20a3-4086-ada8-1ea7a34cad20 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:27.378204 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64186489-4392-4c5c-94fe-3dac482c8cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:27.378809 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64186489-4392-4c5c-94fe-3dac482c8cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:27.379088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64186489-4392-4c5c-94fe-3dac482c8cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:27.379362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64186489-4392-4c5c-94fe-3dac482c8cf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:27.380002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64186489-4392-4c5c-94fe-3dac482c8cf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:27.381625 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64186489-4392-4c5c-94fe-3dac482c8cf3 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:27.391871 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1226/2424] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:27.396317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6323d-a51e-46b6-915f-1383a4eac35b admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:27.407728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6323d-a51e-46b6-915f-1383a4eac35b admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:27.412589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dd1bf-9b46-45d1-9a25-e8fa5692fe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:27.418565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dd1bf-9b46-45d1-9a25-e8fa5692fe78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:27.418565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dd1bf-9b46-45d1-9a25-e8fa5692fe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:27.418565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4dd1bf-9b46-45d1-9a25-e8fa5692fe78 admin admin] PATH_INFO: `/v3/projects/d1bb06502cac4db983a29bdf8c9c6a0a/users/6e6a8a4cf4094b77826d6ee116efcc64/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:27.418565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4dd1bf-9b46-45d1-9a25-e8fa5692fe78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1bb06502cac4db983a29bdf8c9c6a0a, user_id=6e6a8a4cf4094b77826d6ee116efcc64, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:27.418565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4dd1bf-9b46-45d1-9a25-e8fa5692fe78 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:27.429514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1199/2425] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:28:27 2022] PUT /identity/v3/projects/d1bb06502cac4db983a29bdf8c9c6a0a/users/6e6a8a4cf4094b77826d6ee116efcc64/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:27.440796 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcebfa4-77f1-47e2-823d-e7af84ac393f None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:27.441146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcebfa4-77f1-47e2-823d-e7af84ac393f None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:27.441502 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcebfa4-77f1-47e2-823d-e7af84ac393f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:27.470889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-1dcebfa4-77f1-47e2-823d-e7af84ac393f None None] MFA Rules not processed for user `6e6a8a4cf4094b77826d6ee116efcc64`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:27.472103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-1dcebfa4-77f1-47e2-823d-e7af84ac393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:27.541623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1227/2426] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:27.553663 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dd1bf-9b46-45d1-9a25-e8fa5692fe78 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:27.555367 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4c4dd1bf-9b46-45d1-9a25-e8fa5692fe78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:27.597874 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:27.614286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4dd1bf-9b46-45d1-9a25-e8fa5692fe78 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:27.617999 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd64fe7-afa7-400d-8681-7d363489bef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:27.618646 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd64fe7-afa7-400d-8681-7d363489bef5 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:27.618933 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd64fe7-afa7-400d-8681-7d363489bef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:27.619209 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd64fe7-afa7-400d-8681-7d363489bef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:27.628866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd64fe7-afa7-400d-8681-7d363489bef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:27.630768 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd64fe7-afa7-400d-8681-7d363489bef5 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:27.641329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1200/2427] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:27.769290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcebfa4-77f1-47e2-823d-e7af84ac393f None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:27.778630 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcebfa4-77f1-47e2-823d-e7af84ac393f None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:27.782901 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b740f5e-0246-456b-badc-a4b3f93ada99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:27.783492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b740f5e-0246-456b-badc-a4b3f93ada99 service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:27.783770 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b740f5e-0246-456b-badc-a4b3f93ada99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:27.784037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b740f5e-0246-456b-badc-a4b3f93ada99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:27.822808 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:27.838830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b740f5e-0246-456b-badc-a4b3f93ada99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:27.840271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b740f5e-0246-456b-badc-a4b3f93ada99 service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:27.850438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1228/2428] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:28.325937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd64fe7-afa7-400d-8681-7d363489bef5 service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:28.392087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd64fe7-afa7-400d-8681-7d363489bef5 service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:28.396743 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc252bb-48c8-4ae8-96f8-b5a21eac0b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:28.397376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc252bb-48c8-4ae8-96f8-b5a21eac0b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:28.397656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc252bb-48c8-4ae8-96f8-b5a21eac0b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:28.397931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc252bb-48c8-4ae8-96f8-b5a21eac0b4d admin admin] PATH_INFO: `/v3/users/588f7a764c224d9a88baa3a3ed8de864` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:28.399039 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc252bb-48c8-4ae8-96f8-b5a21eac0b4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=588f7a764c224d9a88baa3a3ed8de864)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:28.401069 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc252bb-48c8-4ae8-96f8-b5a21eac0b4d admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:28.587123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8cc252bb-48c8-4ae8-96f8-b5a21eac0b4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '588f7a764c224d9a88baa3a3ed8de864'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:28.587666 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8cc252bb-48c8-4ae8-96f8-b5a21eac0b4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '588f7a764c224d9a88baa3a3ed8de864'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:28.715717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8cc252bb-48c8-4ae8-96f8-b5a21eac0b4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '588f7a764c224d9a88baa3a3ed8de864'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:28.728447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8cc252bb-48c8-4ae8-96f8-b5a21eac0b4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '588f7a764c224d9a88baa3a3ed8de864'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:28.735616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8cc252bb-48c8-4ae8-96f8-b5a21eac0b4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '588f7a764c224d9a88baa3a3ed8de864'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:28.741632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1201/2429] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:28:28 2022] DELETE /identity/v3/users/588f7a764c224d9a88baa3a3ed8de864 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:28.746266 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e8947-b0f2-4476-87bc-170e4b3a8e2b None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:28.746760 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e8947-b0f2-4476-87bc-170e4b3a8e2b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:28.746863 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e8947-b0f2-4476-87bc-170e4b3a8e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:28.766596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f76e8947-b0f2-4476-87bc-170e4b3a8e2b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:28.767821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f76e8947-b0f2-4476-87bc-170e4b3a8e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:28.826325 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1229/2430] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:28.836990 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc252bb-48c8-4ae8-96f8-b5a21eac0b4d admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:28.838457 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8cc252bb-48c8-4ae8-96f8-b5a21eac0b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:28.902099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc252bb-48c8-4ae8-96f8-b5a21eac0b4d admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:28.905708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b1a8e-69d9-4a23-bcf4-3974cb170c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:28.906320 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b1a8e-69d9-4a23-bcf4-3974cb170c15 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:28.906586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b1a8e-69d9-4a23-bcf4-3974cb170c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:28.906859 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b1a8e-69d9-4a23-bcf4-3974cb170c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:28.915707 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b1a8e-69d9-4a23-bcf4-3974cb170c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:28.917159 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772b1a8e-69d9-4a23-bcf4-3974cb170c15 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:28.927562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1202/2431] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:29.324272 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e8947-b0f2-4476-87bc-170e4b3a8e2b None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:29.325864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f76e8947-b0f2-4476-87bc-170e4b3a8e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:29.340118 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e8947-b0f2-4476-87bc-170e4b3a8e2b None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:29.345655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bf067-ef24-4547-b49b-662cc8764bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:29.346288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bf067-ef24-4547-b49b-662cc8764bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:29.346598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bf067-ef24-4547-b49b-662cc8764bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:29.346895 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bf067-ef24-4547-b49b-662cc8764bb2 admin admin] PATH_INFO: `/v3/projects/944825a0f0ee48beab1f0bb7cbb39859` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:29.348179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4bf067-ef24-4547-b49b-662cc8764bb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=944825a0f0ee48beab1f0bb7cbb39859)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:29.351231 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4bf067-ef24-4547-b49b-662cc8764bb2 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:29.468127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-de4bf067-ef24-4547-b49b-662cc8764bb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '944825a0f0ee48beab1f0bb7cbb39859', 'request_id': 'req-de4bf067-ef24-4547-b49b-662cc8764bb2'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:29.474432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-de4bf067-ef24-4547-b49b-662cc8764bb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '944825a0f0ee48beab1f0bb7cbb39859', 'request_id': 'req-de4bf067-ef24-4547-b49b-662cc8764bb2'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:29.481367 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-de4bf067-ef24-4547-b49b-662cc8764bb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '944825a0f0ee48beab1f0bb7cbb39859', 'request_id': 'req-de4bf067-ef24-4547-b49b-662cc8764bb2'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:29.557387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-de4bf067-ef24-4547-b49b-662cc8764bb2 admin admin] The token cache is being invalidate because project 944825a0f0ee48beab1f0bb7cbb39859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:28:29.557940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-de4bf067-ef24-4547-b49b-662cc8764bb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:29.558556 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-de4bf067-ef24-4547-b49b-662cc8764bb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:29.560580 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1230/2432] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:28:29 2022] DELETE /identity/v3/projects/944825a0f0ee48beab1f0bb7cbb39859 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:29.565226 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b1a8e-69d9-4a23-bcf4-3974cb170c15 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:29.570554 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:29.576129 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b1a8e-69d9-4a23-bcf4-3974cb170c15 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:29.644036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ac299-51d8-4145-aff1-e222553a7f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:29.644944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ac299-51d8-4145-aff1-e222553a7f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:29.645234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ac299-51d8-4145-aff1-e222553a7f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:29.645514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ac299-51d8-4145-aff1-e222553a7f0f admin admin] PATH_INFO: `/v3/users/6e6a8a4cf4094b77826d6ee116efcc64` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:29.646660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ac299-51d8-4145-aff1-e222553a7f0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6a8a4cf4094b77826d6ee116efcc64)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:29.648577 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ac299-51d8-4145-aff1-e222553a7f0f admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:29.834105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a54ac299-51d8-4145-aff1-e222553a7f0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6a8a4cf4094b77826d6ee116efcc64'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:29.834764 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a54ac299-51d8-4145-aff1-e222553a7f0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6a8a4cf4094b77826d6ee116efcc64'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:29.887477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a54ac299-51d8-4145-aff1-e222553a7f0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6a8a4cf4094b77826d6ee116efcc64'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:29.901154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a54ac299-51d8-4145-aff1-e222553a7f0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6a8a4cf4094b77826d6ee116efcc64'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:29.909135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a54ac299-51d8-4145-aff1-e222553a7f0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6a8a4cf4094b77826d6ee116efcc64'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:29.915271 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1203/2433] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:28:29 2022] DELETE /identity/v3/users/6e6a8a4cf4094b77826d6ee116efcc64 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:30.380874 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bf067-ef24-4547-b49b-662cc8764bb2 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:30.382433 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-de4bf067-ef24-4547-b49b-662cc8764bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:30.398636 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bf067-ef24-4547-b49b-662cc8764bb2 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:30.456920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd25ce9-2086-41bd-9994-e7729d8b5444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:30.457552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd25ce9-2086-41bd-9994-e7729d8b5444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:30.457832 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd25ce9-2086-41bd-9994-e7729d8b5444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:30.458107 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd25ce9-2086-41bd-9994-e7729d8b5444 admin admin] PATH_INFO: `/v3/projects/d1bb06502cac4db983a29bdf8c9c6a0a` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:30.459197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd25ce9-2086-41bd-9994-e7729d8b5444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1bb06502cac4db983a29bdf8c9c6a0a)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:30.461147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd25ce9-2086-41bd-9994-e7729d8b5444 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:30.467156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:30.579581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-fdd25ce9-2086-41bd-9994-e7729d8b5444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1bb06502cac4db983a29bdf8c9c6a0a', 'request_id': 'req-fdd25ce9-2086-41bd-9994-e7729d8b5444'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:30.584976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-fdd25ce9-2086-41bd-9994-e7729d8b5444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1bb06502cac4db983a29bdf8c9c6a0a', 'request_id': 'req-fdd25ce9-2086-41bd-9994-e7729d8b5444'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:30.591057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-fdd25ce9-2086-41bd-9994-e7729d8b5444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1bb06502cac4db983a29bdf8c9c6a0a', 'request_id': 'req-fdd25ce9-2086-41bd-9994-e7729d8b5444'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:30.669944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-fdd25ce9-2086-41bd-9994-e7729d8b5444 admin admin] The token cache is being invalidate because project d1bb06502cac4db983a29bdf8c9c6a0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:28:30.670263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-fdd25ce9-2086-41bd-9994-e7729d8b5444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:30.670902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-fdd25ce9-2086-41bd-9994-e7729d8b5444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:30.672482 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1231/2434] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:28:30 2022] DELETE /identity/v3/projects/d1bb06502cac4db983a29bdf8c9c6a0a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:30.678836 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f62e6-9e78-40a7-8bde-7f71945b3c10 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:30.679327 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f62e6-9e78-40a7-8bde-7f71945b3c10 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:30.683425 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f62e6-9e78-40a7-8bde-7f71945b3c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:30.700095 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-d76f62e6-9e78-40a7-8bde-7f71945b3c10 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:30.701327 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d76f62e6-9e78-40a7-8bde-7f71945b3c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:30.762836 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1204/2435] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:30.770977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b2404-20a3-4948-83c2-80a781127c47 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:30.771263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b2404-20a3-4948-83c2-80a781127c47 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:30.771539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b2404-20a3-4948-83c2-80a781127c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:30.791304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-055b2404-20a3-4948-83c2-80a781127c47 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:30.792487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-055b2404-20a3-4948-83c2-80a781127c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:30.800465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1232/2436] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:30.805138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f62e6-9e78-40a7-8bde-7f71945b3c10 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:30.816514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f62e6-9e78-40a7-8bde-7f71945b3c10 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:30.822083 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad48b2e-c03b-451a-8b99-bbf87cb34d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:30.822692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad48b2e-c03b-451a-8b99-bbf87cb34d3a admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:30.822989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad48b2e-c03b-451a-8b99-bbf87cb34d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:30.823281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad48b2e-c03b-451a-8b99-bbf87cb34d3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:30.824013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad48b2e-c03b-451a-8b99-bbf87cb34d3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:28:30.824405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad48b2e-c03b-451a-8b99-bbf87cb34d3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:30.826134 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad48b2e-c03b-451a-8b99-bbf87cb34d3a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:30.838690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1205/2437] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:28:30.843599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b2404-20a3-4948-83c2-80a781127c47 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:30.852984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b2404-20a3-4948-83c2-80a781127c47 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:30.857768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ef02b-3d05-43a9-8a49-426de78017a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:30.858387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ef02b-3d05-43a9-8a49-426de78017a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:30.858671 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ef02b-3d05-43a9-8a49-426de78017a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:30.858948 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ef02b-3d05-43a9-8a49-426de78017a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:30.859581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ef02b-3d05-43a9-8a49-426de78017a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:30.861548 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ef02b-3d05-43a9-8a49-426de78017a5 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:30.969940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1233/2438] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:28:30 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:28:30.973945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad48b2e-c03b-451a-8b99-bbf87cb34d3a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:30.984193 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad48b2e-c03b-451a-8b99-bbf87cb34d3a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:30.988901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:31.041353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2b212-8dc2-44e5-8e49-29518ea22545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:31.041947 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2b212-8dc2-44e5-8e49-29518ea22545 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:31.042226 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2b212-8dc2-44e5-8e49-29518ea22545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:31.042499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2b212-8dc2-44e5-8e49-29518ea22545 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:31.043126 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2b212-8dc2-44e5-8e49-29518ea22545 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:31.045103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2b212-8dc2-44e5-8e49-29518ea22545 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:31.098357 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1206/2439] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:28:30 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:28:31.102515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ef02b-3d05-43a9-8a49-426de78017a5 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:31.113322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ef02b-3d05-43a9-8a49-426de78017a5 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:31.117951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a4708-53aa-48b2-9131-bc765f113ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:31.118539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a4708-53aa-48b2-9131-bc765f113ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:31.118844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a4708-53aa-48b2-9131-bc765f113ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:31.119248 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a4708-53aa-48b2-9131-bc765f113ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:31.119869 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a4708-53aa-48b2-9131-bc765f113ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:31.121555 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a4708-53aa-48b2-9131-bc765f113ba1 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:31.131929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1234/2440] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:31.135769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2b212-8dc2-44e5-8e49-29518ea22545 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:31.146322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2b212-8dc2-44e5-8e49-29518ea22545 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:31.150978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d623b-01fa-48e2-81af-0432cffdb94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:31.151568 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d623b-01fa-48e2-81af-0432cffdb94e admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:31.151856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d623b-01fa-48e2-81af-0432cffdb94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:31.152135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d623b-01fa-48e2-81af-0432cffdb94e admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:31.152922 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d623b-01fa-48e2-81af-0432cffdb94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:31.154538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d623b-01fa-48e2-81af-0432cffdb94e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:31.165116 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1207/2441] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:31.169132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a4708-53aa-48b2-9131-bc765f113ba1 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:31.178879 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a4708-53aa-48b2-9131-bc765f113ba1 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:31.183478 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18201e8-7360-4044-a871-2a8344cd01e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:31.184115 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18201e8-7360-4044-a871-2a8344cd01e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:31.184401 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18201e8-7360-4044-a871-2a8344cd01e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:31.184709 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18201e8-7360-4044-a871-2a8344cd01e8 admin admin] PATH_INFO: `/v3/projects/25ad7faeef694e6aaf6e3044b3bf2fe5/users/0991384f6590447485a3388443c23fd1/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:31.199731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18201e8-7360-4044-a871-2a8344cd01e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ad7faeef694e6aaf6e3044b3bf2fe5, user_id=0991384f6590447485a3388443c23fd1, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:31.203179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18201e8-7360-4044-a871-2a8344cd01e8 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:31.281362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1235/2442] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:28:31 2022] PUT /identity/v3/projects/25ad7faeef694e6aaf6e3044b3bf2fe5/users/0991384f6590447485a3388443c23fd1/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:28:31.292973 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317cea9-7fbd-4612-8fba-608b42916680 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:31.293262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317cea9-7fbd-4612-8fba-608b42916680 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:31.293568 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317cea9-7fbd-4612-8fba-608b42916680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:31.321484 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-d317cea9-7fbd-4612-8fba-608b42916680 None None] MFA Rules not processed for user `0991384f6590447485a3388443c23fd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:31.322694 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d317cea9-7fbd-4612-8fba-608b42916680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:31.387098 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1208/2443] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:31.392042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18201e8-7360-4044-a871-2a8344cd01e8 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:31.452854 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18201e8-7360-4044-a871-2a8344cd01e8 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:31.458124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1643574-101e-48a0-89f6-8f100b7e40e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:31.458726 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1643574-101e-48a0-89f6-8f100b7e40e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:31.459011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1643574-101e-48a0-89f6-8f100b7e40e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:31.459288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1643574-101e-48a0-89f6-8f100b7e40e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:31.459908 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1643574-101e-48a0-89f6-8f100b7e40e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:31.461849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1643574-101e-48a0-89f6-8f100b7e40e0 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:31.557109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1236/2444] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:28:31 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:28:31.561178 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317cea9-7fbd-4612-8fba-608b42916680 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:31.572338 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317cea9-7fbd-4612-8fba-608b42916680 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:31.630772 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71bde5-2240-4b9b-9c4a-eeca470efcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:31.631374 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71bde5-2240-4b9b-9c4a-eeca470efcb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:31.631658 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71bde5-2240-4b9b-9c4a-eeca470efcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:31.631931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71bde5-2240-4b9b-9c4a-eeca470efcb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:31.632973 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71bde5-2240-4b9b-9c4a-eeca470efcb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:31.635005 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71bde5-2240-4b9b-9c4a-eeca470efcb6 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:31.721926 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1209/2445] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:28:31 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:28:31.726035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1643574-101e-48a0-89f6-8f100b7e40e0 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:31.736950 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1643574-101e-48a0-89f6-8f100b7e40e0 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:31.741547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ead98-1097-4d6e-a089-97143aec2715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:31.742191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ead98-1097-4d6e-a089-97143aec2715 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:31.742470 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ead98-1097-4d6e-a089-97143aec2715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:31.742745 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ead98-1097-4d6e-a089-97143aec2715 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:31.743378 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ead98-1097-4d6e-a089-97143aec2715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:31.745015 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ead98-1097-4d6e-a089-97143aec2715 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:31.755551 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1237/2446] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:31.759489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71bde5-2240-4b9b-9c4a-eeca470efcb6 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:31.769837 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71bde5-2240-4b9b-9c4a-eeca470efcb6 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:31.774504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230e44f-6a7c-40e4-b15c-b2857a970493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:31.775121 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230e44f-6a7c-40e4-b15c-b2857a970493 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:31.775386 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230e44f-6a7c-40e4-b15c-b2857a970493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:31.775660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230e44f-6a7c-40e4-b15c-b2857a970493 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:31.776323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230e44f-6a7c-40e4-b15c-b2857a970493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:31.777923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230e44f-6a7c-40e4-b15c-b2857a970493 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:31.779257 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:31.789229 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1210/2447] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:31.793631 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ead98-1097-4d6e-a089-97143aec2715 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:31.800976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:31.805168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ead98-1097-4d6e-a089-97143aec2715 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:31.813894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2ce76-d42b-4e1b-a551-9f66d0ca29bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:31.816806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2ce76-d42b-4e1b-a551-9f66d0ca29bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:31.816806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2ce76-d42b-4e1b-a551-9f66d0ca29bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:31.816806 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2ce76-d42b-4e1b-a551-9f66d0ca29bd admin admin] PATH_INFO: `/v3/projects/15481e785c404a6db8d9db06e103c945/users/2c34025a01b240b58b7819baa7b39aad/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:31.838605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2ce76-d42b-4e1b-a551-9f66d0ca29bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=15481e785c404a6db8d9db06e103c945, user_id=2c34025a01b240b58b7819baa7b39aad, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:31.841373 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2ce76-d42b-4e1b-a551-9f66d0ca29bd admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:31.930408 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1238/2448] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:28:31 2022] PUT /identity/v3/projects/15481e785c404a6db8d9db06e103c945/users/2c34025a01b240b58b7819baa7b39aad/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:31.942186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6be8b-a040-48aa-9830-84aa63ef1e02 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:31.942554 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6be8b-a040-48aa-9830-84aa63ef1e02 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:31.942843 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6be8b-a040-48aa-9830-84aa63ef1e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:31.972303 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-08a6be8b-a040-48aa-9830-84aa63ef1e02 None None] MFA Rules not processed for user `2c34025a01b240b58b7819baa7b39aad`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:31.973568 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-08a6be8b-a040-48aa-9830-84aa63ef1e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:32.036715 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1211/2449] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:32.041184 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2ce76-d42b-4e1b-a551-9f66d0ca29bd admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:32.101911 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2ce76-d42b-4e1b-a551-9f66d0ca29bd admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:32.106405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2cf9f-5ada-4fab-ad2f-3ff32162eb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:32.106995 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2cf9f-5ada-4fab-ad2f-3ff32162eb04 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:32.107274 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2cf9f-5ada-4fab-ad2f-3ff32162eb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:32.107546 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2cf9f-5ada-4fab-ad2f-3ff32162eb04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:32.108158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2cf9f-5ada-4fab-ad2f-3ff32162eb04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:32.110269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2cf9f-5ada-4fab-ad2f-3ff32162eb04 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:32.185440 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1239/2450] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:28:32 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:28:32.189773 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6be8b-a040-48aa-9830-84aa63ef1e02 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:32.200544 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6be8b-a040-48aa-9830-84aa63ef1e02 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:32.256465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08114b-cd83-4ba0-977b-97fce76e52c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:32.257235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08114b-cd83-4ba0-977b-97fce76e52c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:32.257572 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08114b-cd83-4ba0-977b-97fce76e52c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:32.257909 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08114b-cd83-4ba0-977b-97fce76e52c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:32.258662 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08114b-cd83-4ba0-977b-97fce76e52c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:32.260992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08114b-cd83-4ba0-977b-97fce76e52c1 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:32.321442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1212/2451] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:28:32 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:28:32.325690 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2cf9f-5ada-4fab-ad2f-3ff32162eb04 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:32.337082 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2cf9f-5ada-4fab-ad2f-3ff32162eb04 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:32.341711 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969fdb2-b7e9-4d06-b024-177f34346d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:32.342330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969fdb2-b7e9-4d06-b024-177f34346d8c admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:32.342616 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969fdb2-b7e9-4d06-b024-177f34346d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:32.343633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969fdb2-b7e9-4d06-b024-177f34346d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:32.344473 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969fdb2-b7e9-4d06-b024-177f34346d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:32.346139 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969fdb2-b7e9-4d06-b024-177f34346d8c admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:32.356578 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1240/2452] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:32.360766 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08114b-cd83-4ba0-977b-97fce76e52c1 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:32.370523 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08114b-cd83-4ba0-977b-97fce76e52c1 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:32.375631 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c9f33-c4b7-4f98-9066-7c9875943e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:32.376270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c9f33-c4b7-4f98-9066-7c9875943e77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:32.376551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c9f33-c4b7-4f98-9066-7c9875943e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:32.376858 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c9f33-c4b7-4f98-9066-7c9875943e77 admin admin] PATH_INFO: `/v3/projects/57074e3c9f91456f95e1e7d7cf57b5cb/users/98d3639478c64096baac3dc64c1d0155/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:32.391937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c9f33-c4b7-4f98-9066-7c9875943e77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57074e3c9f91456f95e1e7d7cf57b5cb, user_id=98d3639478c64096baac3dc64c1d0155, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:32.394588 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c9f33-c4b7-4f98-9066-7c9875943e77 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:32.447311 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1213/2453] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:28:32 2022] PUT /identity/v3/projects/57074e3c9f91456f95e1e7d7cf57b5cb/users/98d3639478c64096baac3dc64c1d0155/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:32.451376 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969fdb2-b7e9-4d06-b024-177f34346d8c admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:32.462347 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969fdb2-b7e9-4d06-b024-177f34346d8c admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:32.519722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c3b7e-8e10-4919-98d7-8ecc0078921b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:32.520331 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c3b7e-8e10-4919-98d7-8ecc0078921b admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:32.520611 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c3b7e-8e10-4919-98d7-8ecc0078921b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:32.520918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c3b7e-8e10-4919-98d7-8ecc0078921b admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:32.521561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c3b7e-8e10-4919-98d7-8ecc0078921b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:32.523154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c3b7e-8e10-4919-98d7-8ecc0078921b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:32.532870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1241/2454] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:32.537545 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c9f33-c4b7-4f98-9066-7c9875943e77 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:32.547480 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c9f33-c4b7-4f98-9066-7c9875943e77 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:32.552125 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24637d6-8083-40b1-9fd6-eebad19481c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:32.553151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24637d6-8083-40b1-9fd6-eebad19481c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:32.553431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24637d6-8083-40b1-9fd6-eebad19481c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:32.553700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24637d6-8083-40b1-9fd6-eebad19481c8 admin admin] PATH_INFO: `/v3/projects/57074e3c9f91456f95e1e7d7cf57b5cb/users/98d3639478c64096baac3dc64c1d0155/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:32.555783 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24637d6-8083-40b1-9fd6-eebad19481c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57074e3c9f91456f95e1e7d7cf57b5cb, user_id=98d3639478c64096baac3dc64c1d0155, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:32.558624 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24637d6-8083-40b1-9fd6-eebad19481c8 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:32.569375 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1214/2455] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:28:32 2022] PUT /identity/v3/projects/57074e3c9f91456f95e1e7d7cf57b5cb/users/98d3639478c64096baac3dc64c1d0155/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:28:32.580392 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3ab2d-39c2-44ee-8769-d4c4822848c6 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:32.580788 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3ab2d-39c2-44ee-8769-d4c4822848c6 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:32.581065 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3ab2d-39c2-44ee-8769-d4c4822848c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:32.609110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-63e3ab2d-39c2-44ee-8769-d4c4822848c6 None None] MFA Rules not processed for user `98d3639478c64096baac3dc64c1d0155`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:32.610290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-63e3ab2d-39c2-44ee-8769-d4c4822848c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:32.650456 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:32.679273 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1242/2456] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:32.690834 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24637d6-8083-40b1-9fd6-eebad19481c8 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:32.692987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d24637d6-8083-40b1-9fd6-eebad19481c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:32.736131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:32.750744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24637d6-8083-40b1-9fd6-eebad19481c8 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:32.754813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414df5b-4576-45fb-94c0-ffaa6e718fb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:32.755439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414df5b-4576-45fb-94c0-ffaa6e718fb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:32.755725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414df5b-4576-45fb-94c0-ffaa6e718fb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:32.755993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414df5b-4576-45fb-94c0-ffaa6e718fb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:32.812847 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414df5b-4576-45fb-94c0-ffaa6e718fb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:32.814355 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414df5b-4576-45fb-94c0-ffaa6e718fb1 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:32.823261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1215/2457] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:35.779188 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3ab2d-39c2-44ee-8769-d4c4822848c6 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:35.789198 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3ab2d-39c2-44ee-8769-d4c4822848c6 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:35.792713 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b3d91-78ab-497e-9ed1-55c238bbdb79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:35.793306 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b3d91-78ab-497e-9ed1-55c238bbdb79 service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:35.793583 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b3d91-78ab-497e-9ed1-55c238bbdb79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:35.793851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b3d91-78ab-497e-9ed1-55c238bbdb79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:35.855222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b3d91-78ab-497e-9ed1-55c238bbdb79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:35.856688 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b3d91-78ab-497e-9ed1-55c238bbdb79 service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:35.865611 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1243/2458] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:37.212132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414df5b-4576-45fb-94c0-ffaa6e718fb1 service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:37.221305 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414df5b-4576-45fb-94c0-ffaa6e718fb1 service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:37.224974 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d5c39-0c17-4b6b-a01b-933e0fafacd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:37.225563 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d5c39-0c17-4b6b-a01b-933e0fafacd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:37.225845 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d5c39-0c17-4b6b-a01b-933e0fafacd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:37.226116 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d5c39-0c17-4b6b-a01b-933e0fafacd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:37.235718 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d5c39-0c17-4b6b-a01b-933e0fafacd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:37.237174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d5c39-0c17-4b6b-a01b-933e0fafacd2 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:37.246398 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1216/2459] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:42.851205 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b3d91-78ab-497e-9ed1-55c238bbdb79 service cinder] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:42.912917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b3d91-78ab-497e-9ed1-55c238bbdb79 service cinder] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:42.917544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e6165-a1ee-4721-81cf-456afc1abe6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:42.918161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e6165-a1ee-4721-81cf-456afc1abe6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:42.918442 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e6165-a1ee-4721-81cf-456afc1abe6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:42.918714 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e6165-a1ee-4721-81cf-456afc1abe6b admin admin] PATH_INFO: `/v3/users/0991384f6590447485a3388443c23fd1` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:42.919823 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e6165-a1ee-4721-81cf-456afc1abe6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0991384f6590447485a3388443c23fd1)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:42.921770 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e6165-a1ee-4721-81cf-456afc1abe6b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:43.106004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d86e6165-a1ee-4721-81cf-456afc1abe6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0991384f6590447485a3388443c23fd1'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:43.230348 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d86e6165-a1ee-4721-81cf-456afc1abe6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0991384f6590447485a3388443c23fd1'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:43.230981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d86e6165-a1ee-4721-81cf-456afc1abe6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0991384f6590447485a3388443c23fd1'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:43.239133 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d86e6165-a1ee-4721-81cf-456afc1abe6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0991384f6590447485a3388443c23fd1'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:43.249649 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:43.251958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d86e6165-a1ee-4721-81cf-456afc1abe6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0991384f6590447485a3388443c23fd1'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:43.258565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1244/2460] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:28:42 2022] DELETE /identity/v3/users/0991384f6590447485a3388443c23fd1 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:43.262874 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4ccb3-afa1-4bf1-b69b-50da7bf44b9e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:43.263360 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4ccb3-afa1-4bf1-b69b-50da7bf44b9e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:43.263476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4ccb3-afa1-4bf1-b69b-50da7bf44b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:43.282678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-28b4ccb3-afa1-4bf1-b69b-50da7bf44b9e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:43.283945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-28b4ccb3-afa1-4bf1-b69b-50da7bf44b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:43.345542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1217/2461] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:43.355245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e6165-a1ee-4721-81cf-456afc1abe6b admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:43.356668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-d86e6165-a1ee-4721-81cf-456afc1abe6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:43.415611 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e6165-a1ee-4721-81cf-456afc1abe6b admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:43.419407 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d378c-0fd6-4903-a575-a9f96f279acd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:43.420059 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d378c-0fd6-4903-a575-a9f96f279acd service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:43.420289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d378c-0fd6-4903-a575-a9f96f279acd service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:43.420562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d378c-0fd6-4903-a575-a9f96f279acd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:43.430302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d378c-0fd6-4903-a575-a9f96f279acd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:43.431773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d378c-0fd6-4903-a575-a9f96f279acd service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:43.441681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1245/2462] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:43.833157 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4ccb3-afa1-4bf1-b69b-50da7bf44b9e None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:43.834616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-28b4ccb3-afa1-4bf1-b69b-50da7bf44b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:43.846853 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4ccb3-afa1-4bf1-b69b-50da7bf44b9e None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:43.851584 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831ac88-eff1-49c2-8ecf-df0e60910533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:43.852223 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831ac88-eff1-49c2-8ecf-df0e60910533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:43.852505 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831ac88-eff1-49c2-8ecf-df0e60910533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:43.852813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831ac88-eff1-49c2-8ecf-df0e60910533 admin admin] PATH_INFO: `/v3/projects/25ad7faeef694e6aaf6e3044b3bf2fe5` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:43.853949 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831ac88-eff1-49c2-8ecf-df0e60910533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25ad7faeef694e6aaf6e3044b3bf2fe5)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:43.855855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831ac88-eff1-49c2-8ecf-df0e60910533 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:43.985704 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f831ac88-eff1-49c2-8ecf-df0e60910533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25ad7faeef694e6aaf6e3044b3bf2fe5', 'request_id': 'req-f831ac88-eff1-49c2-8ecf-df0e60910533'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:43.991758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f831ac88-eff1-49c2-8ecf-df0e60910533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25ad7faeef694e6aaf6e3044b3bf2fe5', 'request_id': 'req-f831ac88-eff1-49c2-8ecf-df0e60910533'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:43.998036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f831ac88-eff1-49c2-8ecf-df0e60910533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25ad7faeef694e6aaf6e3044b3bf2fe5', 'request_id': 'req-f831ac88-eff1-49c2-8ecf-df0e60910533'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:44.070459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f831ac88-eff1-49c2-8ecf-df0e60910533 admin admin] The token cache is being invalidate because project 25ad7faeef694e6aaf6e3044b3bf2fe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:28:44.070745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f831ac88-eff1-49c2-8ecf-df0e60910533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:44.071391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f831ac88-eff1-49c2-8ecf-df0e60910533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:44.072909 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1218/2463] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:28:43 2022] DELETE /identity/v3/projects/25ad7faeef694e6aaf6e3044b3bf2fe5 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:44.076868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d378c-0fd6-4903-a575-a9f96f279acd service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:44.086987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d378c-0fd6-4903-a575-a9f96f279acd service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:44.142633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e47ea-c17e-425d-a349-cbc10f3cdb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:44.143259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e47ea-c17e-425d-a349-cbc10f3cdb08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:44.143738 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e47ea-c17e-425d-a349-cbc10f3cdb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:44.144020 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e47ea-c17e-425d-a349-cbc10f3cdb08 admin admin] PATH_INFO: `/v3/users/2c34025a01b240b58b7819baa7b39aad` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:44.145158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e47ea-c17e-425d-a349-cbc10f3cdb08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c34025a01b240b58b7819baa7b39aad)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:44.147151 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e47ea-c17e-425d-a349-cbc10f3cdb08 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:44.351873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1b2e47ea-c17e-425d-a349-cbc10f3cdb08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c34025a01b240b58b7819baa7b39aad'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:44.418703 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1b2e47ea-c17e-425d-a349-cbc10f3cdb08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c34025a01b240b58b7819baa7b39aad'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:44.419324 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1b2e47ea-c17e-425d-a349-cbc10f3cdb08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c34025a01b240b58b7819baa7b39aad'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:44.427380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1b2e47ea-c17e-425d-a349-cbc10f3cdb08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c34025a01b240b58b7819baa7b39aad'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:44.439130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1b2e47ea-c17e-425d-a349-cbc10f3cdb08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c34025a01b240b58b7819baa7b39aad'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:44.445178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1246/2464] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:28:44 2022] DELETE /identity/v3/users/2c34025a01b240b58b7819baa7b39aad => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:44.838088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831ac88-eff1-49c2-8ecf-df0e60910533 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:44.839663 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f831ac88-eff1-49c2-8ecf-df0e60910533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:44.848591 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:44.853923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831ac88-eff1-49c2-8ecf-df0e60910533 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:44.911044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:44.911673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:44.911955 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:44.912228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c admin admin] PATH_INFO: `/v3/projects/15481e785c404a6db8d9db06e103c945` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:44.913365 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c admin admin] RBAC: Authorizing `identity:delete_project(project_id=15481e785c404a6db8d9db06e103c945)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:44.915290 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:45.043023 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15481e785c404a6db8d9db06e103c945', 'request_id': 'req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:45.049226 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15481e785c404a6db8d9db06e103c945', 'request_id': 'req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:45.055314 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15481e785c404a6db8d9db06e103c945', 'request_id': 'req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:45.155640 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c admin admin] The token cache is being invalidate because project 15481e785c404a6db8d9db06e103c945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:28:45.155903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:45.156531 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:45.158044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1219/2465] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:28:44 2022] DELETE /identity/v3/projects/15481e785c404a6db8d9db06e103c945 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:45.162025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e47ea-c17e-425d-a349-cbc10f3cdb08 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:45.163466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-1b2e47ea-c17e-425d-a349-cbc10f3cdb08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:45.177164 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e47ea-c17e-425d-a349-cbc10f3cdb08 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:45.188427 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:45.234638 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305fab0-37b4-41ff-8644-938452678546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:45.235262 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305fab0-37b4-41ff-8644-938452678546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:45.235545 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305fab0-37b4-41ff-8644-938452678546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:45.235818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305fab0-37b4-41ff-8644-938452678546 admin admin] PATH_INFO: `/v3/users/98d3639478c64096baac3dc64c1d0155` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:45.236949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305fab0-37b4-41ff-8644-938452678546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98d3639478c64096baac3dc64c1d0155)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:45.239024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305fab0-37b4-41ff-8644-938452678546 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:45.427496 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5305fab0-37b4-41ff-8644-938452678546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98d3639478c64096baac3dc64c1d0155'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:45.499194 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5305fab0-37b4-41ff-8644-938452678546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98d3639478c64096baac3dc64c1d0155'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:45.499834 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5305fab0-37b4-41ff-8644-938452678546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98d3639478c64096baac3dc64c1d0155'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:45.508451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5305fab0-37b4-41ff-8644-938452678546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98d3639478c64096baac3dc64c1d0155'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:45.520487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5305fab0-37b4-41ff-8644-938452678546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98d3639478c64096baac3dc64c1d0155'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:45.526431 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1247/2466] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:28:45 2022] DELETE /identity/v3/users/98d3639478c64096baac3dc64c1d0155 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:46.189776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:46.189776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:46.202312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f3264-1afd-4d3a-9a46-f2b8bbcdc04c admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:46.272512 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfa631-6a1e-4177-aa9f-debf631498a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:46.273221 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfa631-6a1e-4177-aa9f-debf631498a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:46.273672 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfa631-6a1e-4177-aa9f-debf631498a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:46.274082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfa631-6a1e-4177-aa9f-debf631498a6 admin admin] PATH_INFO: `/v3/projects/57074e3c9f91456f95e1e7d7cf57b5cb` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:46.275522 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfa631-6a1e-4177-aa9f-debf631498a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57074e3c9f91456f95e1e7d7cf57b5cb)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:46.278125 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfa631-6a1e-4177-aa9f-debf631498a6 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:46.427624 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:46.436768 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6ddfa631-6a1e-4177-aa9f-debf631498a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57074e3c9f91456f95e1e7d7cf57b5cb', 'request_id': 'req-6ddfa631-6a1e-4177-aa9f-debf631498a6'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:46.442353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6ddfa631-6a1e-4177-aa9f-debf631498a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57074e3c9f91456f95e1e7d7cf57b5cb', 'request_id': 'req-6ddfa631-6a1e-4177-aa9f-debf631498a6'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:46.448569 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6ddfa631-6a1e-4177-aa9f-debf631498a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57074e3c9f91456f95e1e7d7cf57b5cb', 'request_id': 'req-6ddfa631-6a1e-4177-aa9f-debf631498a6'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:46.539447 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6ddfa631-6a1e-4177-aa9f-debf631498a6 admin admin] The token cache is being invalidate because project 57074e3c9f91456f95e1e7d7cf57b5cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:28:46.539737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6ddfa631-6a1e-4177-aa9f-debf631498a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:46.540356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6ddfa631-6a1e-4177-aa9f-debf631498a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:46.541917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1220/2467] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:28:46 2022] DELETE /identity/v3/projects/57074e3c9f91456f95e1e7d7cf57b5cb => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:46.548027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128db47-88d0-42c8-b330-93055160dcca None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:46.548530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128db47-88d0-42c8-b330-93055160dcca None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:46.548661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128db47-88d0-42c8-b330-93055160dcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:46.567820 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-3128db47-88d0-42c8-b330-93055160dcca None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:46.569061 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-3128db47-88d0-42c8-b330-93055160dcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:46.630298 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1248/2468] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:46.692699 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42314-e634-42ec-b149-f3435ca6952e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:46.692914 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42314-e634-42ec-b149-f3435ca6952e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:46.693164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42314-e634-42ec-b149-f3435ca6952e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:46.713656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-6ea42314-e634-42ec-b149-f3435ca6952e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:46.714864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6ea42314-e634-42ec-b149-f3435ca6952e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:46.722830 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1221/2469] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:46.732160 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128db47-88d0-42c8-b330-93055160dcca None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:46.742589 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128db47-88d0-42c8-b330-93055160dcca None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:46.746908 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf70195-a4c3-4d85-95a4-7caf9aeb23ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:46.747524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf70195-a4c3-4d85-95a4-7caf9aeb23ee admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:46.747804 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf70195-a4c3-4d85-95a4-7caf9aeb23ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:46.748074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf70195-a4c3-4d85-95a4-7caf9aeb23ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:46.748741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf70195-a4c3-4d85-95a4-7caf9aeb23ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:28:46.749066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf70195-a4c3-4d85-95a4-7caf9aeb23ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:46.750672 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf70195-a4c3-4d85-95a4-7caf9aeb23ee admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:46.762541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1249/2470] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:28:46.766651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea42314-e634-42ec-b149-f3435ca6952e None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:46.776836 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea42314-e634-42ec-b149-f3435ca6952e None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:46.781600 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6ab4d-08eb-4643-be87-de425e067f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:46.782192 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6ab4d-08eb-4643-be87-de425e067f15 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:46.782478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6ab4d-08eb-4643-be87-de425e067f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:46.782752 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6ab4d-08eb-4643-be87-de425e067f15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:46.783385 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6ab4d-08eb-4643-be87-de425e067f15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:46.785339 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6ab4d-08eb-4643-be87-de425e067f15 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:46.877431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1222/2471] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:28:46 2022] POST /identity/v3/projects => generated 399 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:28:46.882293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf70195-a4c3-4d85-95a4-7caf9aeb23ee admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:46.892540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf70195-a4c3-4d85-95a4-7caf9aeb23ee admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:46.911988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:46.949892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a4bad-3653-4c2f-9161-51f20b2df9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:46.950432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a4bad-3653-4c2f-9161-51f20b2df9db admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:46.950717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a4bad-3653-4c2f-9161-51f20b2df9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:46.950992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a4bad-3653-4c2f-9161-51f20b2df9db admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:46.951625 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a4bad-3653-4c2f-9161-51f20b2df9db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:46.953573 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a4bad-3653-4c2f-9161-51f20b2df9db admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:47.065025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1250/2472] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:28:46 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:28:47.069812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6ab4d-08eb-4643-be87-de425e067f15 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:47.080383 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6ab4d-08eb-4643-be87-de425e067f15 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:47.085225 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a626cff-0ff6-4ae6-ac02-16ab8cec303a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:47.085821 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a626cff-0ff6-4ae6-ac02-16ab8cec303a admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:47.086382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a626cff-0ff6-4ae6-ac02-16ab8cec303a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:47.086630 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a626cff-0ff6-4ae6-ac02-16ab8cec303a admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:47.087276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a626cff-0ff6-4ae6-ac02-16ab8cec303a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:47.089670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a626cff-0ff6-4ae6-ac02-16ab8cec303a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:47.100585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1223/2473] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:47.104669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a4bad-3653-4c2f-9161-51f20b2df9db admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:47.115216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a4bad-3653-4c2f-9161-51f20b2df9db admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:47.120072 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e2873-f491-4584-bc4d-ac36096ec095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:47.120752 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e2873-f491-4584-bc4d-ac36096ec095 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:47.123433 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e2873-f491-4584-bc4d-ac36096ec095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:47.123777 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e2873-f491-4584-bc4d-ac36096ec095 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:47.124603 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e2873-f491-4584-bc4d-ac36096ec095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:47.129591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e2873-f491-4584-bc4d-ac36096ec095 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:47.144313 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1251/2474] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:28:47.145761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a626cff-0ff6-4ae6-ac02-16ab8cec303a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:47.156006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a626cff-0ff6-4ae6-ac02-16ab8cec303a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:47.160577 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81db36d-c99d-4609-9ed4-9f786ead028e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:47.170016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81db36d-c99d-4609-9ed4-9f786ead028e admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:47.170016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81db36d-c99d-4609-9ed4-9f786ead028e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:47.170016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81db36d-c99d-4609-9ed4-9f786ead028e admin admin] PATH_INFO: `/v3/projects/c0f808ce716d400faf664c41b721ca73/users/1cd39524f30f4cb298cde363c3cc46bc/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:47.177501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81db36d-c99d-4609-9ed4-9f786ead028e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0f808ce716d400faf664c41b721ca73, user_id=1cd39524f30f4cb298cde363c3cc46bc, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:47.180127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81db36d-c99d-4609-9ed4-9f786ead028e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:47.230490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1224/2475] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:28:47 2022] PUT /identity/v3/projects/c0f808ce716d400faf664c41b721ca73/users/1cd39524f30f4cb298cde363c3cc46bc/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:47.241441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec639bec-c4a3-405a-a3ab-9799cac677b7 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:47.241873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec639bec-c4a3-405a-a3ab-9799cac677b7 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:47.242152 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec639bec-c4a3-405a-a3ab-9799cac677b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:47.269319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-ec639bec-c4a3-405a-a3ab-9799cac677b7 None None] MFA Rules not processed for user `1cd39524f30f4cb298cde363c3cc46bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:47.270512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ec639bec-c4a3-405a-a3ab-9799cac677b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:47.335508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1252/2476] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:47 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:47.352425 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81db36d-c99d-4609-9ed4-9f786ead028e admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:47.352425 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c81db36d-c99d-4609-9ed4-9f786ead028e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:47.410940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81db36d-c99d-4609-9ed4-9f786ead028e admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:47.414645 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a751d5-31cc-41c7-8e9c-6c6b6d16fb6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:47.415256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a751d5-31cc-41c7-8e9c-6c6b6d16fb6c service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:47.415543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a751d5-31cc-41c7-8e9c-6c6b6d16fb6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:47.415820 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a751d5-31cc-41c7-8e9c-6c6b6d16fb6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:47.424111 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a751d5-31cc-41c7-8e9c-6c6b6d16fb6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:47.428063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a751d5-31cc-41c7-8e9c-6c6b6d16fb6c service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:47.435187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1225/2477] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:47 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:55.067430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec639bec-c4a3-405a-a3ab-9799cac677b7 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:55.069557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ec639bec-c4a3-405a-a3ab-9799cac677b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:55.126588 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec639bec-c4a3-405a-a3ab-9799cac677b7 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:55.130163 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b871a-b590-445c-aaca-a4c24bcdaaaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:55.131249 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b871a-b590-445c-aaca-a4c24bcdaaaa service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:55.131710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b871a-b590-445c-aaca-a4c24bcdaaaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:55.131987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b871a-b590-445c-aaca-a4c24bcdaaaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:55.133739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-150b871a-b590-445c-aaca-a4c24bcdaaaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:55.178074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:55.204062 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b871a-b590-445c-aaca-a4c24bcdaaaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:55.205767 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b871a-b590-445c-aaca-a4c24bcdaaaa service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:55.217158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1253/2478] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:56.451635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a751d5-31cc-41c7-8e9c-6c6b6d16fb6c service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:56.453156 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c3a751d5-31cc-41c7-8e9c-6c6b6d16fb6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:56.512184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a751d5-31cc-41c7-8e9c-6c6b6d16fb6c service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:56.515976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced43ba3-841a-4d1b-bfbc-633e7f12aa15 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe9f3fec7104a289e05e88e2d751458', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:56.516738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced43ba3-841a-4d1b-bfbc-633e7f12aa15 service glance] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:56.517026 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced43ba3-841a-4d1b-bfbc-633e7f12aa15 service glance] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:56.517300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced43ba3-841a-4d1b-bfbc-633e7f12aa15 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:56.519089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-ced43ba3-841a-4d1b-bfbc-633e7f12aa15 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:56.530989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:56.533352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced43ba3-841a-4d1b-bfbc-633e7f12aa15 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:56.534836 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced43ba3-841a-4d1b-bfbc-633e7f12aa15 service glance] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:56.546055 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1226/2479] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:57.330588 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b871a-b590-445c-aaca-a4c24bcdaaaa service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:57.406915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b871a-b590-445c-aaca-a4c24bcdaaaa service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:57.412329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91411973-aa5a-44a3-8dec-dca1ea5e628a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:57.412992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91411973-aa5a-44a3-8dec-dca1ea5e628a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:57.413277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91411973-aa5a-44a3-8dec-dca1ea5e628a admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:57.413554 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91411973-aa5a-44a3-8dec-dca1ea5e628a admin admin] PATH_INFO: `/v3/users/1cd39524f30f4cb298cde363c3cc46bc` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:57.414704 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91411973-aa5a-44a3-8dec-dca1ea5e628a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd39524f30f4cb298cde363c3cc46bc)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:57.416630 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91411973-aa5a-44a3-8dec-dca1ea5e628a admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:57.664275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-91411973-aa5a-44a3-8dec-dca1ea5e628a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd39524f30f4cb298cde363c3cc46bc'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:57.757041 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-91411973-aa5a-44a3-8dec-dca1ea5e628a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd39524f30f4cb298cde363c3cc46bc'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:57.757684 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-91411973-aa5a-44a3-8dec-dca1ea5e628a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd39524f30f4cb298cde363c3cc46bc'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:57.765643 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-91411973-aa5a-44a3-8dec-dca1ea5e628a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd39524f30f4cb298cde363c3cc46bc'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:57.779487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-91411973-aa5a-44a3-8dec-dca1ea5e628a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd39524f30f4cb298cde363c3cc46bc'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:57.785847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1254/2480] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:28:57 2022] DELETE /identity/v3/users/1cd39524f30f4cb298cde363c3cc46bc => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:28:57.791114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283ec3c-dbbb-4c5b-8846-685c7c113477 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:57.791614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283ec3c-dbbb-4c5b-8846-685c7c113477 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:57.791825 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283ec3c-dbbb-4c5b-8846-685c7c113477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:57.812726 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-6283ec3c-dbbb-4c5b-8846-685c7c113477 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:28:57.814026 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6283ec3c-dbbb-4c5b-8846-685c7c113477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:57.876802 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1227/2481] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:28:57.887628 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91411973-aa5a-44a3-8dec-dca1ea5e628a admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:57.889333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-91411973-aa5a-44a3-8dec-dca1ea5e628a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:57.979639 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91411973-aa5a-44a3-8dec-dca1ea5e628a admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:57.991261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57881d-bed0-4b56-9987-8b6dc1865091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:57.992072 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57881d-bed0-4b56-9987-8b6dc1865091 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:57.992590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57881d-bed0-4b56-9987-8b6dc1865091 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:57.999288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57881d-bed0-4b56-9987-8b6dc1865091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:58.000197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:28:58.005301 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57881d-bed0-4b56-9987-8b6dc1865091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:58.007810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57881d-bed0-4b56-9987-8b6dc1865091 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:58.029055 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1255/2482] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:28:58.479786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283ec3c-dbbb-4c5b-8846-685c7c113477 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:28:58.481499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6283ec3c-dbbb-4c5b-8846-685c7c113477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:28:58.497209 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283ec3c-dbbb-4c5b-8846-685c7c113477 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:28:58.502266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152f620-7a48-4df1-84fb-454f250d6cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:28:58.508062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152f620-7a48-4df1-84fb-454f250d6cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:28:58.508062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152f620-7a48-4df1-84fb-454f250d6cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:28:58.508062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152f620-7a48-4df1-84fb-454f250d6cf8 admin admin] PATH_INFO: `/v3/projects/c0f808ce716d400faf664c41b721ca73` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:28:58.508062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152f620-7a48-4df1-84fb-454f250d6cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0f808ce716d400faf664c41b721ca73)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:28:58.508062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152f620-7a48-4df1-84fb-454f250d6cf8 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:28:58.604590 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a152f620-7a48-4df1-84fb-454f250d6cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0f808ce716d400faf664c41b721ca73', 'request_id': 'req-a152f620-7a48-4df1-84fb-454f250d6cf8'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:58.611127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a152f620-7a48-4df1-84fb-454f250d6cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0f808ce716d400faf664c41b721ca73', 'request_id': 'req-a152f620-7a48-4df1-84fb-454f250d6cf8'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:58.618650 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a152f620-7a48-4df1-84fb-454f250d6cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0f808ce716d400faf664c41b721ca73', 'request_id': 'req-a152f620-7a48-4df1-84fb-454f250d6cf8'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:58.689962 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a152f620-7a48-4df1-84fb-454f250d6cf8 admin admin] The token cache is being invalidate because project c0f808ce716d400faf664c41b721ca73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:28:58.690784 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a152f620-7a48-4df1-84fb-454f250d6cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:58.691792 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a152f620-7a48-4df1-84fb-454f250d6cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:28:58.694744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1228/2483] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:28:58 2022] DELETE /identity/v3/projects/c0f808ce716d400faf664c41b721ca73 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:29:03.137169 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a604e1-d5ef-4119-a5d0-6d519215ed1d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:29:03.143720 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a604e1-d5ef-4119-a5d0-6d519215ed1d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:29:03.143720 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a604e1-d5ef-4119-a5d0-6d519215ed1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:29:03.160893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-23a604e1-d5ef-4119-a5d0-6d519215ed1d None None] MFA Rules not processed for user `d3f9fac4e1de488b9964f988d238483a`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:29:03.162137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-23a604e1-d5ef-4119-a5d0-6d519215ed1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:03.232322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1256/2484] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:29:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:29:06.073517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152f620-7a48-4df1-84fb-454f250d6cf8 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:29:06.075057 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a152f620-7a48-4df1-84fb-454f250d6cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:06.133708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152f620-7a48-4df1-84fb-454f250d6cf8 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:29:06.137372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3b298-3c63-4a31-9b46-479ec16e65fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:29:06.137985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3b298-3c63-4a31-9b46-479ec16e65fd service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:29:06.138287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3b298-3c63-4a31-9b46-479ec16e65fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:29:06.138556 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3b298-3c63-4a31-9b46-479ec16e65fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:29:06.140266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8ab3b298-3c63-4a31-9b46-479ec16e65fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:06.174323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:29:06.220465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3b298-3c63-4a31-9b46-479ec16e65fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:29:06.222059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3b298-3c63-4a31-9b46-479ec16e65fd service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:29:06.240248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1229/2485] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:29:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:29:06.653453 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a604e1-d5ef-4119-a5d0-6d519215ed1d None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:29:06.654987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-23a604e1-d5ef-4119-a5d0-6d519215ed1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:06.753802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a604e1-d5ef-4119-a5d0-6d519215ed1d None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:29:06.761302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b10a1-02e5-4fb7-809b-f9a4b6d5be0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:29:06.762003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b10a1-02e5-4fb7-809b-f9a4b6d5be0b service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:29:06.762285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b10a1-02e5-4fb7-809b-f9a4b6d5be0b service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:29:06.762558 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b10a1-02e5-4fb7-809b-f9a4b6d5be0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:29:06.764349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-fe1b10a1-02e5-4fb7-809b-f9a4b6d5be0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:06.877850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b10a1-02e5-4fb7-809b-f9a4b6d5be0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:29:06.880060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b10a1-02e5-4fb7-809b-f9a4b6d5be0b service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:29:06.898815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1257/2486] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:29:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:29:08.107626 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acaaf8f-f8d7-4143-9a39-4c543213564f None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:29:08.108125 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acaaf8f-f8d7-4143-9a39-4c543213564f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:29:08.108225 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acaaf8f-f8d7-4143-9a39-4c543213564f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:29:08.143140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-2acaaf8f-f8d7-4143-9a39-4c543213564f None None] MFA Rules not processed for user `0156c689e109489d9265dd3dd4d5cfde`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:29:08.144550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2acaaf8f-f8d7-4143-9a39-4c543213564f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:08.157939 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1230/2487] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:29:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:29:09.535955 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b10a1-02e5-4fb7-809b-f9a4b6d5be0b service nova] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:29:09.537406 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-fe1b10a1-02e5-4fb7-809b-f9a4b6d5be0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:09.547116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:29:09.608428 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b10a1-02e5-4fb7-809b-f9a4b6d5be0b service nova] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:29:09.615121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e61a3-54cc-4fc8-9052-12394429e81c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ee8ba21bc184feeaafacab3e69e87bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:29:09.615799 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e61a3-54cc-4fc8-9052-12394429e81c service placement] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:29:09.616159 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e61a3-54cc-4fc8-9052-12394429e81c service placement] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:29:09.616596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e61a3-54cc-4fc8-9052-12394429e81c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:29:09.619134 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-471e61a3-54cc-4fc8-9052-12394429e81c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:09.634734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e61a3-54cc-4fc8-9052-12394429e81c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:29:09.636437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e61a3-54cc-4fc8-9052-12394429e81c service placement] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:29:09.647295 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1258/2488] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:29:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:29:15.622734 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acaaf8f-f8d7-4143-9a39-4c543213564f None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:29:15.624358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2acaaf8f-f8d7-4143-9a39-4c543213564f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:15.636485 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acaaf8f-f8d7-4143-9a39-4c543213564f None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:29:15.640057 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41d07a-dc0f-4da9-991a-eaf635638c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:29:15.640625 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41d07a-dc0f-4da9-991a-eaf635638c4f service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:29:15.640935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41d07a-dc0f-4da9-991a-eaf635638c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:29:15.641202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41d07a-dc0f-4da9-991a-eaf635638c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:29:15.642838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0c41d07a-dc0f-4da9-991a-eaf635638c4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:15.653075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41d07a-dc0f-4da9-991a-eaf635638c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:29:15.654488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41d07a-dc0f-4da9-991a-eaf635638c4f service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:29:15.662432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1231/2489] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:29:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:29:19.642992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=29 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:29:21.388202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6b236-fea9-45b6-98dd-8dbb9bc934bf None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:29:21.388785 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6b236-fea9-45b6-98dd-8dbb9bc934bf None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:29:21.388785 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6b236-fea9-45b6-98dd-8dbb9bc934bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:29:21.436163 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-64c6b236-fea9-45b6-98dd-8dbb9bc934bf None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:29:21.437379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-64c6b236-fea9-45b6-98dd-8dbb9bc934bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:21.501879 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1259/2490] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Aug 12 12:29:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:29:21.626191 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41d07a-dc0f-4da9-991a-eaf635638c4f service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:29:21.626954 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0c41d07a-dc0f-4da9-991a-eaf635638c4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:21.740127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41d07a-dc0f-4da9-991a-eaf635638c4f service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:29:21.746403 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111db9f6-b801-47f4-9277-6830630f1dfe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:29:21.746958 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111db9f6-b801-47f4-9277-6830630f1dfe service swift] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:29:21.747266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111db9f6-b801-47f4-9277-6830630f1dfe service swift] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:29:21.747538 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111db9f6-b801-47f4-9277-6830630f1dfe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:29:21.756594 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111db9f6-b801-47f4-9277-6830630f1dfe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:29:21.758209 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111db9f6-b801-47f4-9277-6830630f1dfe service swift] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:29:21.766848 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1232/2491] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:29:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 12:29:31.498901 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=40 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:29:31.763286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=78 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:29:39.853897 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6b236-fea9-45b6-98dd-8dbb9bc934bf None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:29:39.863332 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6b236-fea9-45b6-98dd-8dbb9bc934bf None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:29:39.866869 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28663418-cde8-4257-858a-86244eef1017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:29:39.867459 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28663418-cde8-4257-858a-86244eef1017 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:29:39.867743 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28663418-cde8-4257-858a-86244eef1017 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:29:39.868013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28663418-cde8-4257-858a-86244eef1017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:29:39.870507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-28663418-cde8-4257-858a-86244eef1017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:39.879719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28663418-cde8-4257-858a-86244eef1017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:29:39.881190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28663418-cde8-4257-858a-86244eef1017 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:29:39.889708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1260/2492] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:29:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:29:40.225367 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111db9f6-b801-47f4-9277-6830630f1dfe service swift] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:29:40.226917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-111db9f6-b801-47f4-9277-6830630f1dfe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:40.284923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111db9f6-b801-47f4-9277-6830630f1dfe service swift] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:29:40.288436 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8cdcc-e0ec-46c7-a184-00ba1c684c46 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffe9f3fec7104a289e05e88e2d751458', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:29:40.289230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8cdcc-e0ec-46c7-a184-00ba1c684c46 service glance] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:29:40.289511 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8cdcc-e0ec-46c7-a184-00ba1c684c46 service glance] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:29:40.289782 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8cdcc-e0ec-46c7-a184-00ba1c684c46 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:29:40.292102 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a3b8cdcc-e0ec-46c7-a184-00ba1c684c46 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:40.305475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8cdcc-e0ec-46c7-a184-00ba1c684c46 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:29:40.307674 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8cdcc-e0ec-46c7-a184-00ba1c684c46 service glance] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:29:40.321042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1233/2493] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:29:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:29:40.552669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28663418-cde8-4257-858a-86244eef1017 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:29:40.561709 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28663418-cde8-4257-858a-86244eef1017 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:29:40.565382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d383cfa-e0bf-4abd-b606-29fa1d0e1bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:29:40.565979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d383cfa-e0bf-4abd-b606-29fa1d0e1bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:29:40.566258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d383cfa-e0bf-4abd-b606-29fa1d0e1bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:29:40.566529 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d383cfa-e0bf-4abd-b606-29fa1d0e1bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:29:40.568271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-5d383cfa-e0bf-4abd-b606-29fa1d0e1bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:40.583524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d383cfa-e0bf-4abd-b606-29fa1d0e1bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:29:40.583524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d383cfa-e0bf-4abd-b606-29fa1d0e1bd0 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:29:40.588874 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1261/2494] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:29:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:29:40.682153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8cdcc-e0ec-46c7-a184-00ba1c684c46 service glance] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:29:40.691324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8cdcc-e0ec-46c7-a184-00ba1c684c46 service glance] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:29:40.694931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d0c69-ec93-4f20-85a1-cd16933d669c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:29:40.695519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d0c69-ec93-4f20-85a1-cd16933d669c service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:29:40.695803 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d0c69-ec93-4f20-85a1-cd16933d669c service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:29:40.696078 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d0c69-ec93-4f20-85a1-cd16933d669c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:29:40.698166 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0e1d0c69-ec93-4f20-85a1-cd16933d669c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:40.711559 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d0c69-ec93-4f20-85a1-cd16933d669c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:29:40.713918 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d0c69-ec93-4f20-85a1-cd16933d669c service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:29:40.724714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1234/2495] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:29:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:29:43.676334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef1ada-abb1-4f4c-993a-29f0f5ce7ba3 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:29:43.676851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef1ada-abb1-4f4c-993a-29f0f5ce7ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:29:43.676962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef1ada-abb1-4f4c-993a-29f0f5ce7ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:29:43.695641 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-37ef1ada-abb1-4f4c-993a-29f0f5ce7ba3 None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:29:43.696862 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-37ef1ada-abb1-4f4c-993a-29f0f5ce7ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:43.704106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1262/2496] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Aug 12 12:29:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:29:43.716627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d0c69-ec93-4f20-85a1-cd16933d669c service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:29:43.728148 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d0c69-ec93-4f20-85a1-cd16933d669c service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:29:43.733864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649b8db-968c-41b7-b45e-583c8ba1d1f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:29:43.734456 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649b8db-968c-41b7-b45e-583c8ba1d1f9 service swift] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:29:43.734738 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649b8db-968c-41b7-b45e-583c8ba1d1f9 service swift] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:29:43.735010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649b8db-968c-41b7-b45e-583c8ba1d1f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:29:43.743425 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649b8db-968c-41b7-b45e-583c8ba1d1f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:29:43.745158 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649b8db-968c-41b7-b45e-583c8ba1d1f9 service swift] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:29:43.753512 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1235/2497] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:29:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 12 12:29:53.690888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=15 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:29:53.750720 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=38 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:29:59.861130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e09a06-6a27-4d81-a765-39df3181fd73 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:29:59.861583 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e09a06-6a27-4d81-a765-39df3181fd73 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:29:59.861692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e09a06-6a27-4d81-a765-39df3181fd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:29:59.880723 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-b0e09a06-6a27-4d81-a765-39df3181fd73 None None] MFA Rules not processed for user `a90e3591b4144fdfb8b5ed5391aec7c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:29:59.881926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b0e09a06-6a27-4d81-a765-39df3181fd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:29:59.936196 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1263/2498] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:29:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:30:00.241876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649b8db-968c-41b7-b45e-583c8ba1d1f9 service swift] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:30:00.251325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649b8db-968c-41b7-b45e-583c8ba1d1f9 service swift] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:30:00.254912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f70026-ab66-49c9-8c55-e07134cc9b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:30:00.255496 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f70026-ab66-49c9-8c55-e07134cc9b00 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:30:00.255779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f70026-ab66-49c9-8c55-e07134cc9b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:30:00.256051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f70026-ab66-49c9-8c55-e07134cc9b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:30:00.257761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-22f70026-ab66-49c9-8c55-e07134cc9b00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:30:00.267421 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f70026-ab66-49c9-8c55-e07134cc9b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:30:00.268873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f70026-ab66-49c9-8c55-e07134cc9b00 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:30:00.278011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1236/2499] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:30:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:30:05.130159 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2704de-b09e-4f77-ac7c-8807543a9c74 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:30:05.130641 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2704de-b09e-4f77-ac7c-8807543a9c74 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:30:05.130746 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2704de-b09e-4f77-ac7c-8807543a9c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:30:05.151550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-6f2704de-b09e-4f77-ac7c-8807543a9c74 None None] MFA Rules not processed for user `ab4e815bbe7f4ff880f4cc633da1ed8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:30:05.153579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-6f2704de-b09e-4f77-ac7c-8807543a9c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:30:05.162938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1264/2500] 10.222.0.27 () {58 vars in 967 bytes} [Fri Aug 12 12:30:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:30:05.177762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f70026-ab66-49c9-8c55-e07134cc9b00 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:30:05.181233 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2704de-b09e-4f77-ac7c-8807543a9c74 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:30:05.193534 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f70026-ab66-49c9-8c55-e07134cc9b00 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:30:05.193795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2704de-b09e-4f77-ac7c-8807543a9c74 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:30:05.199635 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611faaa-ae7d-4a9b-ad5d-f29d3de4c9ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5123401076a84ffcab49b00fee65d14d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:30:05.200257 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611faaa-ae7d-4a9b-ad5d-f29d3de4c9ee service swift] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:30:05.200541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611faaa-ae7d-4a9b-ad5d-f29d3de4c9ee service swift] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:30:05.200731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846e6df-d967-4c14-9a23-e3ce0c99381c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:30:05.200880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611faaa-ae7d-4a9b-ad5d-f29d3de4c9ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:30:05.201540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846e6df-d967-4c14-9a23-e3ce0c99381c service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:30:05.201880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846e6df-d967-4c14-9a23-e3ce0c99381c service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:30:05.202189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846e6df-d967-4c14-9a23-e3ce0c99381c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:30:05.204203 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f846e6df-d967-4c14-9a23-e3ce0c99381c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:30:05.210544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611faaa-ae7d-4a9b-ad5d-f29d3de4c9ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:30:05.212138 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611faaa-ae7d-4a9b-ad5d-f29d3de4c9ee service swift] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:30:05.218475 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f846e6df-d967-4c14-9a23-e3ce0c99381c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:30:05.220112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f846e6df-d967-4c14-9a23-e3ce0c99381c service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:30:05.221902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1265/2501] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Aug 12 12:30:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 12 12:30:05.244464 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1237/2502] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:30:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:30:15.217943 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=34 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:30:15.228766 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=12 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:30:18.590240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611faaa-ae7d-4a9b-ad5d-f29d3de4c9ee service swift] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:30:18.591707 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0611faaa-ae7d-4a9b-ad5d-f29d3de4c9ee service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:30:18.607994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611faaa-ae7d-4a9b-ad5d-f29d3de4c9ee service swift] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:30:18.613525 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441175d-7356-4509-ac88-6e19e16f36f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:30:18.614062 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441175d-7356-4509-ac88-6e19e16f36f7 service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:30:18.614347 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441175d-7356-4509-ac88-6e19e16f36f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:30:18.614624 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441175d-7356-4509-ac88-6e19e16f36f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:30:18.616340 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f441175d-7356-4509-ac88-6e19e16f36f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:30:18.629312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441175d-7356-4509-ac88-6e19e16f36f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:30:18.631241 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441175d-7356-4509-ac88-6e19e16f36f7 service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:30:18.642179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1266/2503] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:30:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:30:22.421198 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846e6df-d967-4c14-9a23-e3ce0c99381c service nova] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:30:22.430688 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846e6df-d967-4c14-9a23-e3ce0c99381c service nova] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:30:22.434410 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd620cf8-d2a7-42f4-9444-3bb1b03b1c21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:30:22.435110 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd620cf8-d2a7-42f4-9444-3bb1b03b1c21 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:30:22.435477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd620cf8-d2a7-42f4-9444-3bb1b03b1c21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:30:22.435820 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd620cf8-d2a7-42f4-9444-3bb1b03b1c21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:30:22.438096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-cd620cf8-d2a7-42f4-9444-3bb1b03b1c21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:30:22.498039 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd620cf8-d2a7-42f4-9444-3bb1b03b1c21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:30:22.499506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd620cf8-d2a7-42f4-9444-3bb1b03b1c21 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:30:22.509114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1238/2504] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:30:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:30:28.637435 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=6 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:30:29.350813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441175d-7356-4509-ac88-6e19e16f36f7 service nova] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:30:29.361601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441175d-7356-4509-ac88-6e19e16f36f7 service nova] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:30:29.373349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59248284-a055-411a-ba23-7c4d78311ee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:30:29.373349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59248284-a055-411a-ba23-7c4d78311ee8 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:30:29.373349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59248284-a055-411a-ba23-7c4d78311ee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:30:29.373349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59248284-a055-411a-ba23-7c4d78311ee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:30:29.373791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-59248284-a055-411a-ba23-7c4d78311ee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:30:29.438681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59248284-a055-411a-ba23-7c4d78311ee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:30:29.440145 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59248284-a055-411a-ba23-7c4d78311ee8 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:30:29.450271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1267/2505] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:30:32.504486 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=28 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:30:37.068299 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd620cf8-d2a7-42f4-9444-3bb1b03b1c21 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:30:37.070669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-cd620cf8-d2a7-42f4-9444-3bb1b03b1c21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:30:37.086025 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd620cf8-d2a7-42f4-9444-3bb1b03b1c21 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:30:37.091313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b5205-c510-4518-8b10-acdaf9f6dd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:30:37.091997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b5205-c510-4518-8b10-acdaf9f6dd33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:30:37.092283 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b5205-c510-4518-8b10-acdaf9f6dd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:30:37.092561 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b5205-c510-4518-8b10-acdaf9f6dd33 admin admin] PATH_INFO: `/v3/users/803cf2e2fb664442ae8b0f8919a90aec` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:30:37.093752 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b5205-c510-4518-8b10-acdaf9f6dd33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=803cf2e2fb664442ae8b0f8919a90aec)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:30:37.097397 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b5205-c510-4518-8b10-acdaf9f6dd33 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:30:37.239586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-fb9b5205-c510-4518-8b10-acdaf9f6dd33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '803cf2e2fb664442ae8b0f8919a90aec'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:37.240238 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-fb9b5205-c510-4518-8b10-acdaf9f6dd33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '803cf2e2fb664442ae8b0f8919a90aec'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:37.306207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-fb9b5205-c510-4518-8b10-acdaf9f6dd33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '803cf2e2fb664442ae8b0f8919a90aec'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:37.319047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-fb9b5205-c510-4518-8b10-acdaf9f6dd33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '803cf2e2fb664442ae8b0f8919a90aec'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:37.327479 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-fb9b5205-c510-4518-8b10-acdaf9f6dd33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '803cf2e2fb664442ae8b0f8919a90aec'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:37.334277 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1239/2506] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:30:37 2022] DELETE /identity/v3/users/803cf2e2fb664442ae8b0f8919a90aec => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:30:37.599890 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59248284-a055-411a-ba23-7c4d78311ee8 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:30:37.618655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59248284-a055-411a-ba23-7c4d78311ee8 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:30:37.687208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ab356-1030-480d-9643-b64471c429d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:30:37.695921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ab356-1030-480d-9643-b64471c429d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:30:37.695921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ab356-1030-480d-9643-b64471c429d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:30:37.695921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ab356-1030-480d-9643-b64471c429d5 admin admin] PATH_INFO: `/v3/projects/b20b2b17c8794507a7cd8fecfb50e13b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:30:37.695921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ab356-1030-480d-9643-b64471c429d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b20b2b17c8794507a7cd8fecfb50e13b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:30:37.695921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ab356-1030-480d-9643-b64471c429d5 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:30:37.821563 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-988ab356-1030-480d-9643-b64471c429d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b20b2b17c8794507a7cd8fecfb50e13b', 'request_id': 'req-988ab356-1030-480d-9643-b64471c429d5'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:37.827340 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-988ab356-1030-480d-9643-b64471c429d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b20b2b17c8794507a7cd8fecfb50e13b', 'request_id': 'req-988ab356-1030-480d-9643-b64471c429d5'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:37.833455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-988ab356-1030-480d-9643-b64471c429d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b20b2b17c8794507a7cd8fecfb50e13b', 'request_id': 'req-988ab356-1030-480d-9643-b64471c429d5'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:37.906359 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-988ab356-1030-480d-9643-b64471c429d5 admin admin] The token cache is being invalidate because project b20b2b17c8794507a7cd8fecfb50e13b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:30:37.906651 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-988ab356-1030-480d-9643-b64471c429d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:37.907281 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-988ab356-1030-480d-9643-b64471c429d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:37.908785 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1268/2507] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:30:37 2022] DELETE /identity/v3/projects/b20b2b17c8794507a7cd8fecfb50e13b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:30:37.912758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b5205-c510-4518-8b10-acdaf9f6dd33 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:30:37.914174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-fb9b5205-c510-4518-8b10-acdaf9f6dd33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:30:37.927179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b5205-c510-4518-8b10-acdaf9f6dd33 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:30:37.984661 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a411d-d5da-4a2b-92c4-8f221e813104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:30:37.985313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a411d-d5da-4a2b-92c4-8f221e813104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:30:37.985596 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a411d-d5da-4a2b-92c4-8f221e813104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:30:37.985871 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a411d-d5da-4a2b-92c4-8f221e813104 admin admin] PATH_INFO: `/v3/users/e8c4df1815df4411b80b4f4152ddedd1` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:30:37.987003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a411d-d5da-4a2b-92c4-8f221e813104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c4df1815df4411b80b4f4152ddedd1)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:30:37.988957 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a411d-d5da-4a2b-92c4-8f221e813104 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:30:38.170997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8b0a411d-d5da-4a2b-92c4-8f221e813104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c4df1815df4411b80b4f4152ddedd1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:38.171663 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8b0a411d-d5da-4a2b-92c4-8f221e813104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c4df1815df4411b80b4f4152ddedd1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:38.248949 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8b0a411d-d5da-4a2b-92c4-8f221e813104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c4df1815df4411b80b4f4152ddedd1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:38.261432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8b0a411d-d5da-4a2b-92c4-8f221e813104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c4df1815df4411b80b4f4152ddedd1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:38.263442 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:30:38.268670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-8b0a411d-d5da-4a2b-92c4-8f221e813104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c4df1815df4411b80b4f4152ddedd1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:38.274511 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1240/2508] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:30:37 2022] DELETE /identity/v3/users/e8c4df1815df4411b80b4f4152ddedd1 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:30:38.524623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ab356-1030-480d-9643-b64471c429d5 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:30:38.531070 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-988ab356-1030-480d-9643-b64471c429d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:30:38.539867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ab356-1030-480d-9643-b64471c429d5 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:30:38.567175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:30:38.604539 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35175da2-3446-4543-ab06-fbcbb2a4eb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:30:38.605942 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35175da2-3446-4543-ab06-fbcbb2a4eb87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:30:38.606071 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35175da2-3446-4543-ab06-fbcbb2a4eb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:30:38.606257 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35175da2-3446-4543-ab06-fbcbb2a4eb87 admin admin] PATH_INFO: `/v3/projects/0427b62c1d4a4621b1fc8fa8cfa8fea6` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:30:38.607401 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35175da2-3446-4543-ab06-fbcbb2a4eb87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0427b62c1d4a4621b1fc8fa8cfa8fea6)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:30:38.609444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35175da2-3446-4543-ab06-fbcbb2a4eb87 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:30:38.701612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-35175da2-3446-4543-ab06-fbcbb2a4eb87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0427b62c1d4a4621b1fc8fa8cfa8fea6', 'request_id': 'req-35175da2-3446-4543-ab06-fbcbb2a4eb87'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:38.707100 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-35175da2-3446-4543-ab06-fbcbb2a4eb87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0427b62c1d4a4621b1fc8fa8cfa8fea6', 'request_id': 'req-35175da2-3446-4543-ab06-fbcbb2a4eb87'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:38.713550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-35175da2-3446-4543-ab06-fbcbb2a4eb87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0427b62c1d4a4621b1fc8fa8cfa8fea6', 'request_id': 'req-35175da2-3446-4543-ab06-fbcbb2a4eb87'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:38.774222 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-35175da2-3446-4543-ab06-fbcbb2a4eb87 admin admin] The token cache is being invalidate because project 0427b62c1d4a4621b1fc8fa8cfa8fea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:30:38.774437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-35175da2-3446-4543-ab06-fbcbb2a4eb87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:38.775079 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-35175da2-3446-4543-ab06-fbcbb2a4eb87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:30:38.776561 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1269/2509] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:30:38 2022] DELETE /identity/v3/projects/0427b62c1d4a4621b1fc8fa8cfa8fea6 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:30:48.363179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=4 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:30:48.849448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:23.318051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a411d-d5da-4a2b-92c4-8f221e813104 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:23.319486 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8b0a411d-d5da-4a2b-92c4-8f221e813104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:23.377300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a411d-d5da-4a2b-92c4-8f221e813104 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:23.426291 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd21a7c-6209-4e2d-babe-8e4725fc125d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:23.427657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd21a7c-6209-4e2d-babe-8e4725fc125d service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:23.427848 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd21a7c-6209-4e2d-babe-8e4725fc125d service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:23.428176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd21a7c-6209-4e2d-babe-8e4725fc125d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:23.429905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1bd21a7c-6209-4e2d-babe-8e4725fc125d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:23.492983 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd21a7c-6209-4e2d-babe-8e4725fc125d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:23.494425 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd21a7c-6209-4e2d-babe-8e4725fc125d service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:23.504050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1241/2510] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:31:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:31:27.652918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35175da2-3446-4543-ab06-fbcbb2a4eb87 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:27.654931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-35175da2-3446-4543-ab06-fbcbb2a4eb87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:27.671304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35175da2-3446-4543-ab06-fbcbb2a4eb87 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:27.677564 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fdeef-d17b-4f9f-b5c9-c4300d613c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:27.678477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fdeef-d17b-4f9f-b5c9-c4300d613c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:27.679988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fdeef-d17b-4f9f-b5c9-c4300d613c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:27.680479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fdeef-d17b-4f9f-b5c9-c4300d613c09 admin admin] PATH_INFO: `/v3/users/383cbf83186747a5b81a12ed5288c39a` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:27.681778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fdeef-d17b-4f9f-b5c9-c4300d613c09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=383cbf83186747a5b81a12ed5288c39a)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:27.683872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fdeef-d17b-4f9f-b5c9-c4300d613c09 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:27.840326 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b32fdeef-d17b-4f9f-b5c9-c4300d613c09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '383cbf83186747a5b81a12ed5288c39a'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:27.923451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b32fdeef-d17b-4f9f-b5c9-c4300d613c09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '383cbf83186747a5b81a12ed5288c39a'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:27.924161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b32fdeef-d17b-4f9f-b5c9-c4300d613c09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '383cbf83186747a5b81a12ed5288c39a'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:27.932555 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b32fdeef-d17b-4f9f-b5c9-c4300d613c09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '383cbf83186747a5b81a12ed5288c39a'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:27.947224 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-b32fdeef-d17b-4f9f-b5c9-c4300d613c09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '383cbf83186747a5b81a12ed5288c39a'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:27.953899 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1270/2511] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:31:27 2022] DELETE /identity/v3/users/383cbf83186747a5b81a12ed5288c39a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:28.275444 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd21a7c-6209-4e2d-babe-8e4725fc125d service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:28.292615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd21a7c-6209-4e2d-babe-8e4725fc125d service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:28.399305 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b98f9-f8e9-4d54-97ea-cdd33d3600ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:28.400334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b98f9-f8e9-4d54-97ea-cdd33d3600ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:28.400850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b98f9-f8e9-4d54-97ea-cdd33d3600ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:28.401125 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b98f9-f8e9-4d54-97ea-cdd33d3600ea admin admin] PATH_INFO: `/v3/projects/f84d5684e1904e569a5551b9e21ce0ff` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:28.402276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b98f9-f8e9-4d54-97ea-cdd33d3600ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=f84d5684e1904e569a5551b9e21ce0ff)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:28.404373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b98f9-f8e9-4d54-97ea-cdd33d3600ea admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:28.413533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:28.542437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-906b98f9-f8e9-4d54-97ea-cdd33d3600ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f84d5684e1904e569a5551b9e21ce0ff', 'request_id': 'req-906b98f9-f8e9-4d54-97ea-cdd33d3600ea'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:28.548973 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-906b98f9-f8e9-4d54-97ea-cdd33d3600ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f84d5684e1904e569a5551b9e21ce0ff', 'request_id': 'req-906b98f9-f8e9-4d54-97ea-cdd33d3600ea'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:28.558536 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-906b98f9-f8e9-4d54-97ea-cdd33d3600ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f84d5684e1904e569a5551b9e21ce0ff', 'request_id': 'req-906b98f9-f8e9-4d54-97ea-cdd33d3600ea'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:28.633486 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-906b98f9-f8e9-4d54-97ea-cdd33d3600ea admin admin] The token cache is being invalidate because project f84d5684e1904e569a5551b9e21ce0ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:31:28.634070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-906b98f9-f8e9-4d54-97ea-cdd33d3600ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:28.635236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-906b98f9-f8e9-4d54-97ea-cdd33d3600ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:28.636965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1242/2512] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:31:28 2022] DELETE /identity/v3/projects/f84d5684e1904e569a5551b9e21ce0ff => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:31:28.641680 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fccadd-5c1e-4320-9517-1886b21dbf05 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:28.642244 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fccadd-5c1e-4320-9517-1886b21dbf05 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:28.642591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fccadd-5c1e-4320-9517-1886b21dbf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:28.662998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-08fccadd-5c1e-4320-9517-1886b21dbf05 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:28.664326 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-08fccadd-5c1e-4320-9517-1886b21dbf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:28.729827 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1271/2513] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:28.738642 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb0213-d046-4dfa-9d73-aac7cfb1d1ac None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:28.739023 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb0213-d046-4dfa-9d73-aac7cfb1d1ac None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:28.739459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb0213-d046-4dfa-9d73-aac7cfb1d1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:28.760103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-49cb0213-d046-4dfa-9d73-aac7cfb1d1ac None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:28.761420 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-49cb0213-d046-4dfa-9d73-aac7cfb1d1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:28.769521 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1243/2514] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:31:28.774398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fccadd-5c1e-4320-9517-1886b21dbf05 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:28.784213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fccadd-5c1e-4320-9517-1886b21dbf05 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:28.789011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be849665-217c-4d52-a44d-4f974acfe3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:28.789615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be849665-217c-4d52-a44d-4f974acfe3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:28.789891 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be849665-217c-4d52-a44d-4f974acfe3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:28.790346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be849665-217c-4d52-a44d-4f974acfe3f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:28.791035 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be849665-217c-4d52-a44d-4f974acfe3f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:31:28.791341 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be849665-217c-4d52-a44d-4f974acfe3f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:28.792961 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be849665-217c-4d52-a44d-4f974acfe3f7 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:28.807474 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1272/2515] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:31:28.811684 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb0213-d046-4dfa-9d73-aac7cfb1d1ac None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:28.822459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb0213-d046-4dfa-9d73-aac7cfb1d1ac None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:28.835317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d238d-597f-42a4-802d-24b2cc00195e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:28.835317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d238d-597f-42a4-802d-24b2cc00195e admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:28.835317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d238d-597f-42a4-802d-24b2cc00195e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:28.835317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d238d-597f-42a4-802d-24b2cc00195e admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:28.835730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d238d-597f-42a4-802d-24b2cc00195e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:28.835730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d238d-597f-42a4-802d-24b2cc00195e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:28.922047 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1244/2516] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:31:28 2022] POST /identity/v3/projects => generated 403 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:31:28.928109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be849665-217c-4d52-a44d-4f974acfe3f7 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:28.943079 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be849665-217c-4d52-a44d-4f974acfe3f7 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:29.018419 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dff1e9-0bb7-4274-a503-81cc623def14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:29.022261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dff1e9-0bb7-4274-a503-81cc623def14 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:29.022261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dff1e9-0bb7-4274-a503-81cc623def14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:29.022261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dff1e9-0bb7-4274-a503-81cc623def14 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:29.022261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dff1e9-0bb7-4274-a503-81cc623def14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:29.022261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dff1e9-0bb7-4274-a503-81cc623def14 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:29.030515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:29.096873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1273/2517] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:31:28 2022] POST /identity/v3/users => generated 302 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:31:29.100247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d238d-597f-42a4-802d-24b2cc00195e admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:29.111650 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d238d-597f-42a4-802d-24b2cc00195e admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:29.117239 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593a44e-a03a-40f3-adeb-cbf1c94c6f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:29.117912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593a44e-a03a-40f3-adeb-cbf1c94c6f87 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:29.118192 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593a44e-a03a-40f3-adeb-cbf1c94c6f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:29.118491 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593a44e-a03a-40f3-adeb-cbf1c94c6f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:29.119196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593a44e-a03a-40f3-adeb-cbf1c94c6f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:29.120843 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593a44e-a03a-40f3-adeb-cbf1c94c6f87 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:29.132357 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1245/2518] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:31:29.137405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dff1e9-0bb7-4274-a503-81cc623def14 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:29.147553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dff1e9-0bb7-4274-a503-81cc623def14 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:29.152918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce84f2-8068-4b9b-87c5-2871dacfdf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:29.153526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce84f2-8068-4b9b-87c5-2871dacfdf14 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:29.153807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce84f2-8068-4b9b-87c5-2871dacfdf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:29.154077 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce84f2-8068-4b9b-87c5-2871dacfdf14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:29.154734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce84f2-8068-4b9b-87c5-2871dacfdf14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:29.156375 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce84f2-8068-4b9b-87c5-2871dacfdf14 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:29.168668 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1274/2519] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:31:29.172444 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593a44e-a03a-40f3-adeb-cbf1c94c6f87 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:29.183048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593a44e-a03a-40f3-adeb-cbf1c94c6f87 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:29.187740 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527c3c7-ab34-4683-aaa6-2d2855b83720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:29.188378 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527c3c7-ab34-4683-aaa6-2d2855b83720 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:29.188681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527c3c7-ab34-4683-aaa6-2d2855b83720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:29.188964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527c3c7-ab34-4683-aaa6-2d2855b83720 admin admin] PATH_INFO: `/v3/projects/1cbfcad76e9d406d9ad492976390aef9/users/2805c504bf2040d7b4ad7794aada8d2f/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:29.204756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527c3c7-ab34-4683-aaa6-2d2855b83720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cbfcad76e9d406d9ad492976390aef9, user_id=2805c504bf2040d7b4ad7794aada8d2f, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:29.207401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527c3c7-ab34-4683-aaa6-2d2855b83720 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:29.283766 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1246/2520] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:29 2022] PUT /identity/v3/projects/1cbfcad76e9d406d9ad492976390aef9/users/2805c504bf2040d7b4ad7794aada8d2f/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:29.294246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777024c1-b6aa-4b4e-ac15-246c35f43519 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:29.294584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777024c1-b6aa-4b4e-ac15-246c35f43519 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:29.294884 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777024c1-b6aa-4b4e-ac15-246c35f43519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:29.323857 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-777024c1-b6aa-4b4e-ac15-246c35f43519 None None] MFA Rules not processed for user `2805c504bf2040d7b4ad7794aada8d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:29.325210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-777024c1-b6aa-4b4e-ac15-246c35f43519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:29.397064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1275/2521] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:29 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:29.401207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527c3c7-ab34-4683-aaa6-2d2855b83720 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:29.469095 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527c3c7-ab34-4683-aaa6-2d2855b83720 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:29.474388 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280181ca-631d-494a-a77b-25991109c767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:29.475012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280181ca-631d-494a-a77b-25991109c767 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:29.475293 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280181ca-631d-494a-a77b-25991109c767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:29.475562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280181ca-631d-494a-a77b-25991109c767 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:29.476231 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280181ca-631d-494a-a77b-25991109c767 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:29.478172 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280181ca-631d-494a-a77b-25991109c767 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:29.570816 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1247/2522] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:31:29 2022] POST /identity/v3/projects => generated 403 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:31:29.575000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777024c1-b6aa-4b4e-ac15-246c35f43519 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:29.589917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777024c1-b6aa-4b4e-ac15-246c35f43519 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:29.644591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877b371-a84f-4469-9420-daa3da294282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:29.645837 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877b371-a84f-4469-9420-daa3da294282 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:29.646139 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877b371-a84f-4469-9420-daa3da294282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:29.646429 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877b371-a84f-4469-9420-daa3da294282 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:29.647081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e877b371-a84f-4469-9420-daa3da294282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:29.649269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e877b371-a84f-4469-9420-daa3da294282 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:29.723316 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1276/2523] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:31:29 2022] POST /identity/v3/users => generated 308 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:31:29.727449 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280181ca-631d-494a-a77b-25991109c767 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:29.738555 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280181ca-631d-494a-a77b-25991109c767 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:29.743977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f43e1-57a6-4e2c-8ecb-7c16e44a58c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:29.744609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f43e1-57a6-4e2c-8ecb-7c16e44a58c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:29.744952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f43e1-57a6-4e2c-8ecb-7c16e44a58c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:29.745256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f43e1-57a6-4e2c-8ecb-7c16e44a58c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:29.745921 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f43e1-57a6-4e2c-8ecb-7c16e44a58c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:29.747604 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f43e1-57a6-4e2c-8ecb-7c16e44a58c2 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:29.759550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1248/2524] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:31:29.763756 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877b371-a84f-4469-9420-daa3da294282 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:29.774446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877b371-a84f-4469-9420-daa3da294282 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:29.779405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f866f6-009c-4647-8afd-17c632e1c397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:29.780087 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f866f6-009c-4647-8afd-17c632e1c397 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:29.780383 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f866f6-009c-4647-8afd-17c632e1c397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:29.780710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f866f6-009c-4647-8afd-17c632e1c397 admin admin] PATH_INFO: `/v3/projects/4ba05a76e28b4be58f882477edd5cf11/users/e6217cbfb564431ab0b7618695d73c37/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:29.797729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f866f6-009c-4647-8afd-17c632e1c397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ba05a76e28b4be58f882477edd5cf11, user_id=e6217cbfb564431ab0b7618695d73c37, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:29.800416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f866f6-009c-4647-8afd-17c632e1c397 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:29.906553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1277/2525] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:29 2022] PUT /identity/v3/projects/4ba05a76e28b4be58f882477edd5cf11/users/e6217cbfb564431ab0b7618695d73c37/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:31:29.910547 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f43e1-57a6-4e2c-8ecb-7c16e44a58c2 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:29.920396 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f43e1-57a6-4e2c-8ecb-7c16e44a58c2 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:29.923902 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:29.978077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d796a-5a4c-4012-9ae0-f5a8b6f596b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:29.978670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d796a-5a4c-4012-9ae0-f5a8b6f596b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:29.978948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d796a-5a4c-4012-9ae0-f5a8b6f596b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:29.979216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d796a-5a4c-4012-9ae0-f5a8b6f596b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:29.979854 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d796a-5a4c-4012-9ae0-f5a8b6f596b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:29.981463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d796a-5a4c-4012-9ae0-f5a8b6f596b1 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:29.991982 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1249/2526] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:31:29.996285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f866f6-009c-4647-8afd-17c632e1c397 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:30.007524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f866f6-009c-4647-8afd-17c632e1c397 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:30.014214 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d79da-9879-4a15-98d1-2c92b0f61987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:30.014214 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d79da-9879-4a15-98d1-2c92b0f61987 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:30.014214 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d79da-9879-4a15-98d1-2c92b0f61987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:30.014214 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d79da-9879-4a15-98d1-2c92b0f61987 admin admin] PATH_INFO: `/v3/projects/4ba05a76e28b4be58f882477edd5cf11/users/e6217cbfb564431ab0b7618695d73c37/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:30.015158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d79da-9879-4a15-98d1-2c92b0f61987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ba05a76e28b4be58f882477edd5cf11, user_id=e6217cbfb564431ab0b7618695d73c37, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:30.017801 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d79da-9879-4a15-98d1-2c92b0f61987 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:30.028022 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1278/2527] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:29 2022] PUT /identity/v3/projects/4ba05a76e28b4be58f882477edd5cf11/users/e6217cbfb564431ab0b7618695d73c37/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:30.038121 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08b81f-7a85-49e6-808f-c051d20ffa53 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:30.038410 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08b81f-7a85-49e6-808f-c051d20ffa53 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:30.038714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08b81f-7a85-49e6-808f-c051d20ffa53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:30.066777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-ec08b81f-7a85-49e6-808f-c051d20ffa53 None None] MFA Rules not processed for user `e6217cbfb564431ab0b7618695d73c37`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:30.069048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-ec08b81f-7a85-49e6-808f-c051d20ffa53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:30.152511 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1250/2528] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:30 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:30.165612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d79da-9879-4a15-98d1-2c92b0f61987 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:30.167589 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f27d79da-9879-4a15-98d1-2c92b0f61987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:30.195613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:30.235381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d79da-9879-4a15-98d1-2c92b0f61987 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:30.240981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53770d4-55af-4024-9a9a-05ce200b2b97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:30.241599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53770d4-55af-4024-9a9a-05ce200b2b97 service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:30.241877 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53770d4-55af-4024-9a9a-05ce200b2b97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:30.242149 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53770d4-55af-4024-9a9a-05ce200b2b97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:30.251970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53770d4-55af-4024-9a9a-05ce200b2b97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:30.253442 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53770d4-55af-4024-9a9a-05ce200b2b97 service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:30.264281 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1279/2529] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:31:30 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:31:30.413645 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08b81f-7a85-49e6-808f-c051d20ffa53 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:30.477679 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08b81f-7a85-49e6-808f-c051d20ffa53 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:30.482489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851a079-1802-4827-ad8d-5500ae539aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:30.483142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851a079-1802-4827-ad8d-5500ae539aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:30.483452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851a079-1802-4827-ad8d-5500ae539aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:30.483728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851a079-1802-4827-ad8d-5500ae539aea admin admin] PATH_INFO: `/v3/users/2805c504bf2040d7b4ad7794aada8d2f` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:30.484963 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851a079-1802-4827-ad8d-5500ae539aea admin admin] RBAC: Authorizing `identity:delete_user(user_id=2805c504bf2040d7b4ad7794aada8d2f)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:30.487014 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851a079-1802-4827-ad8d-5500ae539aea admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:30.764379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:30.778517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b851a079-1802-4827-ad8d-5500ae539aea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2805c504bf2040d7b4ad7794aada8d2f'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:30.779322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b851a079-1802-4827-ad8d-5500ae539aea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2805c504bf2040d7b4ad7794aada8d2f'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:30.849945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b851a079-1802-4827-ad8d-5500ae539aea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2805c504bf2040d7b4ad7794aada8d2f'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:30.863828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b851a079-1802-4827-ad8d-5500ae539aea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2805c504bf2040d7b4ad7794aada8d2f'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:30.871683 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-b851a079-1802-4827-ad8d-5500ae539aea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2805c504bf2040d7b4ad7794aada8d2f'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:30.877985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1251/2530] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:31:30 2022] DELETE /identity/v3/users/2805c504bf2040d7b4ad7794aada8d2f => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:30.883208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44cb37-cb34-4411-961e-ba4fb9d7398c None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:30.883666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44cb37-cb34-4411-961e-ba4fb9d7398c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:30.883810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44cb37-cb34-4411-961e-ba4fb9d7398c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:30.903713 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-9a44cb37-cb34-4411-961e-ba4fb9d7398c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:30.905033 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9a44cb37-cb34-4411-961e-ba4fb9d7398c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:30.965482 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1280/2531] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:30.975781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851a079-1802-4827-ad8d-5500ae539aea admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:30.977223 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b851a079-1802-4827-ad8d-5500ae539aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:31.034867 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851a079-1802-4827-ad8d-5500ae539aea admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:31.039272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4911e8-43aa-4b32-bdf2-64f386622e05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:31.039870 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4911e8-43aa-4b32-bdf2-64f386622e05 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:31.040155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4911e8-43aa-4b32-bdf2-64f386622e05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:31.040431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4911e8-43aa-4b32-bdf2-64f386622e05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:31.049584 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4911e8-43aa-4b32-bdf2-64f386622e05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:31.051005 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4911e8-43aa-4b32-bdf2-64f386622e05 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:31.061151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1252/2532] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:31:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:31:31.641391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44cb37-cb34-4411-961e-ba4fb9d7398c None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:31.643104 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9a44cb37-cb34-4411-961e-ba4fb9d7398c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:31.656248 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44cb37-cb34-4411-961e-ba4fb9d7398c None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:31.662938 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508670a-b52a-4302-94e0-c1d22408f967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:31.663729 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508670a-b52a-4302-94e0-c1d22408f967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:31.664013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508670a-b52a-4302-94e0-c1d22408f967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:31.664287 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508670a-b52a-4302-94e0-c1d22408f967 admin admin] PATH_INFO: `/v3/projects/1cbfcad76e9d406d9ad492976390aef9` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:31.665690 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1508670a-b52a-4302-94e0-c1d22408f967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cbfcad76e9d406d9ad492976390aef9)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:31.667616 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1508670a-b52a-4302-94e0-c1d22408f967 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:31.785326 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1508670a-b52a-4302-94e0-c1d22408f967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cbfcad76e9d406d9ad492976390aef9', 'request_id': 'req-1508670a-b52a-4302-94e0-c1d22408f967'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:31.791541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1508670a-b52a-4302-94e0-c1d22408f967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cbfcad76e9d406d9ad492976390aef9', 'request_id': 'req-1508670a-b52a-4302-94e0-c1d22408f967'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:31.808775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1508670a-b52a-4302-94e0-c1d22408f967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cbfcad76e9d406d9ad492976390aef9', 'request_id': 'req-1508670a-b52a-4302-94e0-c1d22408f967'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:31.883111 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1508670a-b52a-4302-94e0-c1d22408f967 admin admin] The token cache is being invalidate because project 1cbfcad76e9d406d9ad492976390aef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:31:31.883406 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1508670a-b52a-4302-94e0-c1d22408f967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:31.884179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1508670a-b52a-4302-94e0-c1d22408f967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:31.885713 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1281/2533] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:31:31 2022] DELETE /identity/v3/projects/1cbfcad76e9d406d9ad492976390aef9 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:31.889676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4911e8-43aa-4b32-bdf2-64f386622e05 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:31.900287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4911e8-43aa-4b32-bdf2-64f386622e05 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:31.982516 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff3c4b-a78a-4219-8687-f7deca9733a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:31.983384 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff3c4b-a78a-4219-8687-f7deca9733a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:31.983784 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff3c4b-a78a-4219-8687-f7deca9733a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:31.984175 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff3c4b-a78a-4219-8687-f7deca9733a3 admin admin] PATH_INFO: `/v3/users/e6217cbfb564431ab0b7618695d73c37` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:31.985602 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff3c4b-a78a-4219-8687-f7deca9733a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6217cbfb564431ab0b7618695d73c37)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:31.987724 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff3c4b-a78a-4219-8687-f7deca9733a3 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:32.167537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-80ff3c4b-a78a-4219-8687-f7deca9733a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6217cbfb564431ab0b7618695d73c37'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:32.168214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-80ff3c4b-a78a-4219-8687-f7deca9733a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6217cbfb564431ab0b7618695d73c37'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:32.172113 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:32.266636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-80ff3c4b-a78a-4219-8687-f7deca9733a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6217cbfb564431ab0b7618695d73c37'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:32.281149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-80ff3c4b-a78a-4219-8687-f7deca9733a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6217cbfb564431ab0b7618695d73c37'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:32.288551 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-80ff3c4b-a78a-4219-8687-f7deca9733a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6217cbfb564431ab0b7618695d73c37'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:32.294555 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1253/2534] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:31:31 2022] DELETE /identity/v3/users/e6217cbfb564431ab0b7618695d73c37 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:32.970779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508670a-b52a-4302-94e0-c1d22408f967 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:32.973625 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-1508670a-b52a-4302-94e0-c1d22408f967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:33.008378 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508670a-b52a-4302-94e0-c1d22408f967 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:33.063509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:33.076968 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb987f11-bfef-48de-9b19-24fe52eb10c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:33.080362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb987f11-bfef-48de-9b19-24fe52eb10c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:33.080362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb987f11-bfef-48de-9b19-24fe52eb10c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:33.080362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb987f11-bfef-48de-9b19-24fe52eb10c8 admin admin] PATH_INFO: `/v3/projects/4ba05a76e28b4be58f882477edd5cf11` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:33.080362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb987f11-bfef-48de-9b19-24fe52eb10c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ba05a76e28b4be58f882477edd5cf11)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:33.081432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb987f11-bfef-48de-9b19-24fe52eb10c8 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:33.215168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-bb987f11-bfef-48de-9b19-24fe52eb10c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ba05a76e28b4be58f882477edd5cf11', 'request_id': 'req-bb987f11-bfef-48de-9b19-24fe52eb10c8'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:33.220705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-bb987f11-bfef-48de-9b19-24fe52eb10c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ba05a76e28b4be58f882477edd5cf11', 'request_id': 'req-bb987f11-bfef-48de-9b19-24fe52eb10c8'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:33.230974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-bb987f11-bfef-48de-9b19-24fe52eb10c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ba05a76e28b4be58f882477edd5cf11', 'request_id': 'req-bb987f11-bfef-48de-9b19-24fe52eb10c8'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:33.340675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-bb987f11-bfef-48de-9b19-24fe52eb10c8 admin admin] The token cache is being invalidate because project 4ba05a76e28b4be58f882477edd5cf11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:31:33.341566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-bb987f11-bfef-48de-9b19-24fe52eb10c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:33.342377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-bb987f11-bfef-48de-9b19-24fe52eb10c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:33.344105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1282/2535] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:31:32 2022] DELETE /identity/v3/projects/4ba05a76e28b4be58f882477edd5cf11 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:33.349026 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f6e2e-fe56-42d0-acd8-75d9fad2a446 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:33.358135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f6e2e-fe56-42d0-acd8-75d9fad2a446 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:33.358135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f6e2e-fe56-42d0-acd8-75d9fad2a446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:33.369118 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-710f6e2e-fe56-42d0-acd8-75d9fad2a446 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:33.370339 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-710f6e2e-fe56-42d0-acd8-75d9fad2a446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:33.434788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1254/2536] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:33.444603 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e95d76-85d0-4681-a63f-a17c859bb529 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:33.444929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e95d76-85d0-4681-a63f-a17c859bb529 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:33.445209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e95d76-85d0-4681-a63f-a17c859bb529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:33.466843 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-b6e95d76-85d0-4681-a63f-a17c859bb529 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:33.468055 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b6e95d76-85d0-4681-a63f-a17c859bb529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:33.478007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1283/2537] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:31:33.480564 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f6e2e-fe56-42d0-acd8-75d9fad2a446 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:33.491318 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f6e2e-fe56-42d0-acd8-75d9fad2a446 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:33.495908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ebc6d-aa42-4243-9f50-a2b129262161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:33.496504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ebc6d-aa42-4243-9f50-a2b129262161 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:33.496821 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ebc6d-aa42-4243-9f50-a2b129262161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:33.497094 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ebc6d-aa42-4243-9f50-a2b129262161 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:33.497735 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ebc6d-aa42-4243-9f50-a2b129262161 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:31:33.498039 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ebc6d-aa42-4243-9f50-a2b129262161 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:33.499624 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ebc6d-aa42-4243-9f50-a2b129262161 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:33.513525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1255/2538] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:31:33.518003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e95d76-85d0-4681-a63f-a17c859bb529 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:33.529586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e95d76-85d0-4681-a63f-a17c859bb529 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:33.534590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ceecd9-29b1-4cba-b304-200e9ef5facf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:33.535403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ceecd9-29b1-4cba-b304-200e9ef5facf admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:33.535706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ceecd9-29b1-4cba-b304-200e9ef5facf admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:33.536145 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ceecd9-29b1-4cba-b304-200e9ef5facf admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:33.536892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ceecd9-29b1-4cba-b304-200e9ef5facf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:33.541417 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ceecd9-29b1-4cba-b304-200e9ef5facf admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:33.617505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1284/2539] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:31:33 2022] POST /identity/v3/projects => generated 363 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:31:33.621920 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ebc6d-aa42-4243-9f50-a2b129262161 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:33.633373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ebc6d-aa42-4243-9f50-a2b129262161 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:33.702033 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d95d2-85e1-4c52-b8ee-1168cee033ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:33.703242 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d95d2-85e1-4c52-b8ee-1168cee033ec admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:33.703530 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d95d2-85e1-4c52-b8ee-1168cee033ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:33.703807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d95d2-85e1-4c52-b8ee-1168cee033ec admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:33.704449 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d95d2-85e1-4c52-b8ee-1168cee033ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:33.706742 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d95d2-85e1-4c52-b8ee-1168cee033ec admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:33.834556 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1256/2540] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:31:33 2022] POST /identity/v3/users => generated 282 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:31:33.847148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ceecd9-29b1-4cba-b304-200e9ef5facf admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:33.858883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ceecd9-29b1-4cba-b304-200e9ef5facf admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:33.863591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38517481-0f2f-40bb-b055-97e99b19d0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:33.869000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38517481-0f2f-40bb-b055-97e99b19d0ae admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:33.869000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38517481-0f2f-40bb-b055-97e99b19d0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:33.869000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38517481-0f2f-40bb-b055-97e99b19d0ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:33.869000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38517481-0f2f-40bb-b055-97e99b19d0ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:33.869000 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38517481-0f2f-40bb-b055-97e99b19d0ae admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:33.883004 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1285/2541] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:31:33.884239 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d95d2-85e1-4c52-b8ee-1168cee033ec admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:33.894566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d95d2-85e1-4c52-b8ee-1168cee033ec admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:33.899790 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d18a90-f09e-43f9-ac43-3e857131840e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:33.900383 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d18a90-f09e-43f9-ac43-3e857131840e admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:33.900709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d18a90-f09e-43f9-ac43-3e857131840e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:33.903376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d18a90-f09e-43f9-ac43-3e857131840e admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:33.903376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d18a90-f09e-43f9-ac43-3e857131840e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:33.904048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d18a90-f09e-43f9-ac43-3e857131840e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:33.917461 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1257/2542] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:31:33.921723 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38517481-0f2f-40bb-b055-97e99b19d0ae admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:33.933738 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38517481-0f2f-40bb-b055-97e99b19d0ae admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:33.938363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22dfa4f-8939-4815-a5e9-78d57e5be63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:33.939005 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22dfa4f-8939-4815-a5e9-78d57e5be63c admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:33.939311 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22dfa4f-8939-4815-a5e9-78d57e5be63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:33.939591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22dfa4f-8939-4815-a5e9-78d57e5be63c admin admin] PATH_INFO: `/v3/projects/d803006ec9a240be8c4c43d7d3843ca3/users/ac1814a155c643e68f62b33b97cbda22/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:33.953953 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22dfa4f-8939-4815-a5e9-78d57e5be63c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d803006ec9a240be8c4c43d7d3843ca3, user_id=ac1814a155c643e68f62b33b97cbda22, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:33.956519 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22dfa4f-8939-4815-a5e9-78d57e5be63c admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:34.014310 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1286/2543] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:33 2022] PUT /identity/v3/projects/d803006ec9a240be8c4c43d7d3843ca3/users/ac1814a155c643e68f62b33b97cbda22/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:31:34.024813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040f20e-f7a1-497e-9926-4774bb70cc5e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:34.025127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040f20e-f7a1-497e-9926-4774bb70cc5e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:34.025406 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040f20e-f7a1-497e-9926-4774bb70cc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:34.054022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-5040f20e-f7a1-497e-9926-4774bb70cc5e None None] MFA Rules not processed for user `ac1814a155c643e68f62b33b97cbda22`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:34.055244 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-5040f20e-f7a1-497e-9926-4774bb70cc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:34.078881 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:34.118611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1258/2544] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:34.123015 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22dfa4f-8939-4815-a5e9-78d57e5be63c admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:34.183996 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22dfa4f-8939-4815-a5e9-78d57e5be63c admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:34.189771 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35459e42-0cda-4eda-8688-fa5a50efdcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:34.190376 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35459e42-0cda-4eda-8688-fa5a50efdcbc admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:34.190657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35459e42-0cda-4eda-8688-fa5a50efdcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:34.190926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35459e42-0cda-4eda-8688-fa5a50efdcbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:34.191545 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35459e42-0cda-4eda-8688-fa5a50efdcbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:34.193603 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35459e42-0cda-4eda-8688-fa5a50efdcbc admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:34.289779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:34.300448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1287/2545] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:31:34 2022] POST /identity/v3/projects => generated 365 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:31:34.301560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040f20e-f7a1-497e-9926-4774bb70cc5e None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:34.312540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040f20e-f7a1-497e-9926-4774bb70cc5e None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:34.371161 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3004f7-70ac-4508-b26b-06525f80ad54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:34.371825 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3004f7-70ac-4508-b26b-06525f80ad54 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:34.372042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3004f7-70ac-4508-b26b-06525f80ad54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:34.372314 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3004f7-70ac-4508-b26b-06525f80ad54 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:34.373054 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3004f7-70ac-4508-b26b-06525f80ad54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:34.374961 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3004f7-70ac-4508-b26b-06525f80ad54 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:34.454830 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1259/2546] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:31:34 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:31:34.459234 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35459e42-0cda-4eda-8688-fa5a50efdcbc admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:34.469710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35459e42-0cda-4eda-8688-fa5a50efdcbc admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:34.474962 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c37d50-09b2-4bd5-bd30-ca21bd31c04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:34.475554 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c37d50-09b2-4bd5-bd30-ca21bd31c04b admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:34.475838 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c37d50-09b2-4bd5-bd30-ca21bd31c04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:34.476112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c37d50-09b2-4bd5-bd30-ca21bd31c04b admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:34.476798 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c37d50-09b2-4bd5-bd30-ca21bd31c04b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:34.478424 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c37d50-09b2-4bd5-bd30-ca21bd31c04b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:34.493254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1288/2547] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:31:34.494450 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3004f7-70ac-4508-b26b-06525f80ad54 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:34.504813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3004f7-70ac-4508-b26b-06525f80ad54 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:34.509395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76615c91-656f-4ddd-b68d-5caf166168df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:34.510121 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76615c91-656f-4ddd-b68d-5caf166168df admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:34.510405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76615c91-656f-4ddd-b68d-5caf166168df admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:34.510677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76615c91-656f-4ddd-b68d-5caf166168df admin admin] PATH_INFO: `/v3/projects/eff31dd0e51d423587ae9a49457bfae5/users/174ee9a40726451ab2a4fe24b9f873ef/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:34.527660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76615c91-656f-4ddd-b68d-5caf166168df admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff31dd0e51d423587ae9a49457bfae5, user_id=174ee9a40726451ab2a4fe24b9f873ef, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:34.530415 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76615c91-656f-4ddd-b68d-5caf166168df admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:34.637625 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1260/2548] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:34 2022] PUT /identity/v3/projects/eff31dd0e51d423587ae9a49457bfae5/users/174ee9a40726451ab2a4fe24b9f873ef/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:31:34.641586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c37d50-09b2-4bd5-bd30-ca21bd31c04b admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:34.652119 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c37d50-09b2-4bd5-bd30-ca21bd31c04b admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:34.710269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fc25e-c4b4-4370-b899-6aa9a78613b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:34.710823 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fc25e-c4b4-4370-b899-6aa9a78613b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:34.711125 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fc25e-c4b4-4370-b899-6aa9a78613b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:34.711398 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fc25e-c4b4-4370-b899-6aa9a78613b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:34.712043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fc25e-c4b4-4370-b899-6aa9a78613b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:34.713686 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fc25e-c4b4-4370-b899-6aa9a78613b2 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:34.723858 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1289/2549] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:31:34.728217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76615c91-656f-4ddd-b68d-5caf166168df admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:34.738451 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76615c91-656f-4ddd-b68d-5caf166168df admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:34.743808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72d523-a649-4078-8503-51cc603f1c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:34.744445 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72d523-a649-4078-8503-51cc603f1c5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:34.744756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72d523-a649-4078-8503-51cc603f1c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:34.745035 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72d523-a649-4078-8503-51cc603f1c5f admin admin] PATH_INFO: `/v3/projects/eff31dd0e51d423587ae9a49457bfae5/users/174ee9a40726451ab2a4fe24b9f873ef/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:34.746922 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72d523-a649-4078-8503-51cc603f1c5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff31dd0e51d423587ae9a49457bfae5, user_id=174ee9a40726451ab2a4fe24b9f873ef, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:34.749611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72d523-a649-4078-8503-51cc603f1c5f admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:34.759970 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1261/2550] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:34 2022] PUT /identity/v3/projects/eff31dd0e51d423587ae9a49457bfae5/users/174ee9a40726451ab2a4fe24b9f873ef/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:34.770138 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823c977-a289-4145-8d36-3b50f21582ab None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:34.770619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823c977-a289-4145-8d36-3b50f21582ab None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:34.770934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823c977-a289-4145-8d36-3b50f21582ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:34.798178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-1823c977-a289-4145-8d36-3b50f21582ab None None] MFA Rules not processed for user `174ee9a40726451ab2a4fe24b9f873ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:34.799390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-1823c977-a289-4145-8d36-3b50f21582ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:34.870250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1290/2551] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:34.882418 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72d523-a649-4078-8503-51cc603f1c5f admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:34.885319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2c72d523-a649-4078-8503-51cc603f1c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:34.945896 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72d523-a649-4078-8503-51cc603f1c5f admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:34.949664 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24533c5-b515-4aca-8681-7ca1606c155f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:34.950350 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24533c5-b515-4aca-8681-7ca1606c155f service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:34.950704 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24533c5-b515-4aca-8681-7ca1606c155f service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:34.950981 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24533c5-b515-4aca-8681-7ca1606c155f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:34.960842 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24533c5-b515-4aca-8681-7ca1606c155f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:34.962267 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24533c5-b515-4aca-8681-7ca1606c155f service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:34.972250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1262/2552] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:31:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:31:35.171865 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823c977-a289-4145-8d36-3b50f21582ab None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:35.200544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823c977-a289-4145-8d36-3b50f21582ab None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:35.208420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef819e6-3e68-41c7-8bbc-e12ca56d6c9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:35.208420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef819e6-3e68-41c7-8bbc-e12ca56d6c9c service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:35.208420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef819e6-3e68-41c7-8bbc-e12ca56d6c9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:35.208420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef819e6-3e68-41c7-8bbc-e12ca56d6c9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:35.266650 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef819e6-3e68-41c7-8bbc-e12ca56d6c9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:35.268016 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef819e6-3e68-41c7-8bbc-e12ca56d6c9c service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:35.278054 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1291/2553] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:31:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:31:37.063631 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24533c5-b515-4aca-8681-7ca1606c155f service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:37.065131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c24533c5-b515-4aca-8681-7ca1606c155f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:37.087150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:37.140587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24533c5-b515-4aca-8681-7ca1606c155f service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:37.147154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4057d6a-0330-42a0-b13e-69621cfb95cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:37.147154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4057d6a-0330-42a0-b13e-69621cfb95cd service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:37.147154 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4057d6a-0330-42a0-b13e-69621cfb95cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:37.147609 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4057d6a-0330-42a0-b13e-69621cfb95cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:37.150614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a4057d6a-0330-42a0-b13e-69621cfb95cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:37.218124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4057d6a-0330-42a0-b13e-69621cfb95cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:37.224287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4057d6a-0330-42a0-b13e-69621cfb95cd service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:37.231553 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1263/2554] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:31:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:31:39.921125 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef819e6-3e68-41c7-8bbc-e12ca56d6c9c service cinder] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:39.922583 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9ef819e6-3e68-41c7-8bbc-e12ca56d6c9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:39.936209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef819e6-3e68-41c7-8bbc-e12ca56d6c9c service cinder] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:39.940018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a4b1e-1a1f-48e6-a99b-7152179ea24a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:39.941196 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a4b1e-1a1f-48e6-a99b-7152179ea24a service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:39.941347 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a4b1e-1a1f-48e6-a99b-7152179ea24a service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:39.941934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a4b1e-1a1f-48e6-a99b-7152179ea24a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:39.944064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-518a4b1e-1a1f-48e6-a99b-7152179ea24a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:39.953774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:39.958150 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a4b1e-1a1f-48e6-a99b-7152179ea24a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:39.959981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a4b1e-1a1f-48e6-a99b-7152179ea24a service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:39.974190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1292/2555] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:31:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:31:41.529259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4057d6a-0330-42a0-b13e-69621cfb95cd service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:41.529259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a4057d6a-0330-42a0-b13e-69621cfb95cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:41.544457 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4057d6a-0330-42a0-b13e-69621cfb95cd service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:41.549809 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888d530-d41e-4c9f-bb78-aa0d2fda5fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:41.550800 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888d530-d41e-4c9f-bb78-aa0d2fda5fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:41.551351 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888d530-d41e-4c9f-bb78-aa0d2fda5fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:41.552521 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888d530-d41e-4c9f-bb78-aa0d2fda5fa4 admin admin] PATH_INFO: `/v3/users/0156c689e109489d9265dd3dd4d5cfde` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:41.554106 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888d530-d41e-4c9f-bb78-aa0d2fda5fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0156c689e109489d9265dd3dd4d5cfde)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:41.556092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888d530-d41e-4c9f-bb78-aa0d2fda5fa4 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:41.777874 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2888d530-d41e-4c9f-bb78-aa0d2fda5fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0156c689e109489d9265dd3dd4d5cfde'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:41.778467 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2888d530-d41e-4c9f-bb78-aa0d2fda5fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0156c689e109489d9265dd3dd4d5cfde'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:41.849978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2888d530-d41e-4c9f-bb78-aa0d2fda5fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0156c689e109489d9265dd3dd4d5cfde'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:41.863234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2888d530-d41e-4c9f-bb78-aa0d2fda5fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0156c689e109489d9265dd3dd4d5cfde'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:41.870710 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2888d530-d41e-4c9f-bb78-aa0d2fda5fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0156c689e109489d9265dd3dd4d5cfde'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:41.876769 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1264/2556] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:31:41 2022] DELETE /identity/v3/users/0156c689e109489d9265dd3dd4d5cfde => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:42.222052 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a4b1e-1a1f-48e6-a99b-7152179ea24a service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:42.237344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a4b1e-1a1f-48e6-a99b-7152179ea24a service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:42.296832 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8ee57-e740-41b4-b9f7-3d4e462c6547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:42.297392 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8ee57-e740-41b4-b9f7-3d4e462c6547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:42.297675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8ee57-e740-41b4-b9f7-3d4e462c6547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:42.297953 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8ee57-e740-41b4-b9f7-3d4e462c6547 admin admin] PATH_INFO: `/v3/projects/3d79773c49d643fc8eb04d69aafb0b01` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:42.299071 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8ee57-e740-41b4-b9f7-3d4e462c6547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d79773c49d643fc8eb04d69aafb0b01)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:42.301014 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8ee57-e740-41b4-b9f7-3d4e462c6547 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:42.447864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4dd8ee57-e740-41b4-b9f7-3d4e462c6547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d79773c49d643fc8eb04d69aafb0b01', 'request_id': 'req-4dd8ee57-e740-41b4-b9f7-3d4e462c6547'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:42.454679 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4dd8ee57-e740-41b4-b9f7-3d4e462c6547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d79773c49d643fc8eb04d69aafb0b01', 'request_id': 'req-4dd8ee57-e740-41b4-b9f7-3d4e462c6547'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:42.464775 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4dd8ee57-e740-41b4-b9f7-3d4e462c6547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d79773c49d643fc8eb04d69aafb0b01', 'request_id': 'req-4dd8ee57-e740-41b4-b9f7-3d4e462c6547'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:42.540793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4dd8ee57-e740-41b4-b9f7-3d4e462c6547 admin admin] The token cache is being invalidate because project 3d79773c49d643fc8eb04d69aafb0b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:31:42.541208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4dd8ee57-e740-41b4-b9f7-3d4e462c6547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:42.541936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-4dd8ee57-e740-41b4-b9f7-3d4e462c6547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:42.543749 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1293/2557] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:31:42 2022] DELETE /identity/v3/projects/3d79773c49d643fc8eb04d69aafb0b01 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:42.549283 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffacabc-5064-4649-8043-b94718c9eefa None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:42.550142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffacabc-5064-4649-8043-b94718c9eefa None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:42.550495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffacabc-5064-4649-8043-b94718c9eefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:42.573501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-1ffacabc-5064-4649-8043-b94718c9eefa None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:42.574922 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1ffacabc-5064-4649-8043-b94718c9eefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:42.621266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:42.642925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1265/2558] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:42.652499 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98539444-ef69-4615-911b-0312634885f4 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:42.652947 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98539444-ef69-4615-911b-0312634885f4 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:42.653342 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98539444-ef69-4615-911b-0312634885f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:42.673666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-98539444-ef69-4615-911b-0312634885f4 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:42.674887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-98539444-ef69-4615-911b-0312634885f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:42.682906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1294/2559] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:42.687420 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffacabc-5064-4649-8043-b94718c9eefa None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:42.697391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffacabc-5064-4649-8043-b94718c9eefa None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:42.702423 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32aad9-154c-49c0-8e9a-a5df898b1848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:42.703065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32aad9-154c-49c0-8e9a-a5df898b1848 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:42.703354 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32aad9-154c-49c0-8e9a-a5df898b1848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:42.705019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32aad9-154c-49c0-8e9a-a5df898b1848 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:42.705019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32aad9-154c-49c0-8e9a-a5df898b1848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:31:42.705019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32aad9-154c-49c0-8e9a-a5df898b1848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:42.706564 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32aad9-154c-49c0-8e9a-a5df898b1848 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:42.719471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1266/2560] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:31:42.727710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98539444-ef69-4615-911b-0312634885f4 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:42.738350 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98539444-ef69-4615-911b-0312634885f4 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:42.743225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d8fa0-135b-4d23-98ac-efe19a539929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:42.743836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d8fa0-135b-4d23-98ac-efe19a539929 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:42.744119 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d8fa0-135b-4d23-98ac-efe19a539929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:42.744394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d8fa0-135b-4d23-98ac-efe19a539929 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:42.745058 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d8fa0-135b-4d23-98ac-efe19a539929 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:42.746996 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d8fa0-135b-4d23-98ac-efe19a539929 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:42.906758 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1295/2561] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:31:42 2022] POST /identity/v3/projects => generated 383 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:31:42.907979 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32aad9-154c-49c0-8e9a-a5df898b1848 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:42.918170 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32aad9-154c-49c0-8e9a-a5df898b1848 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:42.981330 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461de1ee-96bc-4e86-8143-8d62c2fef2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:42.981937 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461de1ee-96bc-4e86-8143-8d62c2fef2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:42.982217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461de1ee-96bc-4e86-8143-8d62c2fef2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:42.982490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461de1ee-96bc-4e86-8143-8d62c2fef2f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:42.983117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461de1ee-96bc-4e86-8143-8d62c2fef2f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:42.985180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461de1ee-96bc-4e86-8143-8d62c2fef2f1 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:43.076155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1267/2562] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:31:42 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:31:43.080368 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d8fa0-135b-4d23-98ac-efe19a539929 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:43.093070 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d8fa0-135b-4d23-98ac-efe19a539929 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:43.099044 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdfa45-89a3-4f9b-943f-c4bdd13036fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:43.099821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdfa45-89a3-4f9b-943f-c4bdd13036fd admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:43.100180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdfa45-89a3-4f9b-943f-c4bdd13036fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:43.100835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdfa45-89a3-4f9b-943f-c4bdd13036fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:43.101381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efdfa45-89a3-4f9b-943f-c4bdd13036fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:43.103538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efdfa45-89a3-4f9b-943f-c4bdd13036fd admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:43.115632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1296/2563] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:31:43.119263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461de1ee-96bc-4e86-8143-8d62c2fef2f1 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:43.132119 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461de1ee-96bc-4e86-8143-8d62c2fef2f1 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:43.138204 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b094a3-7c46-492e-b807-79a1f5028250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:43.138811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b094a3-7c46-492e-b807-79a1f5028250 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:43.139095 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b094a3-7c46-492e-b807-79a1f5028250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:43.139367 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b094a3-7c46-492e-b807-79a1f5028250 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:43.140180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b094a3-7c46-492e-b807-79a1f5028250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:43.141827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b094a3-7c46-492e-b807-79a1f5028250 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:43.154852 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1268/2564] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:31:43.159039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdfa45-89a3-4f9b-943f-c4bdd13036fd admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:43.170821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdfa45-89a3-4f9b-943f-c4bdd13036fd admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:43.175605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d44835-fe45-46d2-b3f2-f0c2f8a3ab0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:43.177288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d44835-fe45-46d2-b3f2-f0c2f8a3ab0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:43.177669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d44835-fe45-46d2-b3f2-f0c2f8a3ab0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:43.178009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d44835-fe45-46d2-b3f2-f0c2f8a3ab0a admin admin] PATH_INFO: `/v3/projects/3666c83d26314e4d885a782a0d473a80/users/a1224f280408487391fbe28d50a789c7/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:43.193907 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d44835-fe45-46d2-b3f2-f0c2f8a3ab0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3666c83d26314e4d885a782a0d473a80, user_id=a1224f280408487391fbe28d50a789c7, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:43.196621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d44835-fe45-46d2-b3f2-f0c2f8a3ab0a admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:43.278444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1297/2565] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:43 2022] PUT /identity/v3/projects/3666c83d26314e4d885a782a0d473a80/users/a1224f280408487391fbe28d50a789c7/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:43.290052 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e278bed-7a00-4133-906e-83df0ad7d615 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:43.290340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e278bed-7a00-4133-906e-83df0ad7d615 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:43.290615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e278bed-7a00-4133-906e-83df0ad7d615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:43.322586 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-5e278bed-7a00-4133-906e-83df0ad7d615 None None] MFA Rules not processed for user `a1224f280408487391fbe28d50a789c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:43.323690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-5e278bed-7a00-4133-906e-83df0ad7d615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:43.413618 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1269/2566] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:43.421885 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d44835-fe45-46d2-b3f2-f0c2f8a3ab0a admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:43.424169 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-87d44835-fe45-46d2-b3f2-f0c2f8a3ab0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:43.453305 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:43.520784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d44835-fe45-46d2-b3f2-f0c2f8a3ab0a admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:43.531689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9311ac-3771-4512-a725-ec69f15c38ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:43.541209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9311ac-3771-4512-a725-ec69f15c38ff service nova] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:43.541209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9311ac-3771-4512-a725-ec69f15c38ff service nova] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:43.541209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9311ac-3771-4512-a725-ec69f15c38ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:43.545590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9311ac-3771-4512-a725-ec69f15c38ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:43.547284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9311ac-3771-4512-a725-ec69f15c38ff service nova] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:43.570146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1298/2567] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:31:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:31:44.114951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e278bed-7a00-4133-906e-83df0ad7d615 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:44.116501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-5e278bed-7a00-4133-906e-83df0ad7d615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:44.161625 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:44.187454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e278bed-7a00-4133-906e-83df0ad7d615 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:44.192307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bff795-365f-4359-bbf9-817e83b5b301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:44.193222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bff795-365f-4359-bbf9-817e83b5b301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:44.193507 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bff795-365f-4359-bbf9-817e83b5b301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:44.193786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bff795-365f-4359-bbf9-817e83b5b301 admin admin] PATH_INFO: `/v3/users/d3f9fac4e1de488b9964f988d238483a` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:44.208867 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bff795-365f-4359-bbf9-817e83b5b301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3f9fac4e1de488b9964f988d238483a)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:44.211374 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bff795-365f-4359-bbf9-817e83b5b301 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:44.421546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-14bff795-365f-4359-bbf9-817e83b5b301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3f9fac4e1de488b9964f988d238483a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:44.422788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-14bff795-365f-4359-bbf9-817e83b5b301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3f9fac4e1de488b9964f988d238483a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:44.517636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-14bff795-365f-4359-bbf9-817e83b5b301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3f9fac4e1de488b9964f988d238483a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:44.530945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-14bff795-365f-4359-bbf9-817e83b5b301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3f9fac4e1de488b9964f988d238483a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:44.538147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-14bff795-365f-4359-bbf9-817e83b5b301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3f9fac4e1de488b9964f988d238483a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:44.544012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1270/2568] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:31:44 2022] DELETE /identity/v3/users/d3f9fac4e1de488b9964f988d238483a => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:44.776619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9311ac-3771-4512-a725-ec69f15c38ff service nova] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:44.788026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9311ac-3771-4512-a725-ec69f15c38ff service nova] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:44.848275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c476758-8eee-435e-a9cd-e3682f8f9f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:44.848942 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c476758-8eee-435e-a9cd-e3682f8f9f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:44.853071 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c476758-8eee-435e-a9cd-e3682f8f9f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:44.853071 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c476758-8eee-435e-a9cd-e3682f8f9f6b admin admin] PATH_INFO: `/v3/projects/c0fbadb872e74c57813900e5ebb6ac88` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:44.859733 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c476758-8eee-435e-a9cd-e3682f8f9f6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0fbadb872e74c57813900e5ebb6ac88)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:44.861753 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c476758-8eee-435e-a9cd-e3682f8f9f6b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:44.969593 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1c476758-8eee-435e-a9cd-e3682f8f9f6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0fbadb872e74c57813900e5ebb6ac88', 'request_id': 'req-1c476758-8eee-435e-a9cd-e3682f8f9f6b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:44.975328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1c476758-8eee-435e-a9cd-e3682f8f9f6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0fbadb872e74c57813900e5ebb6ac88', 'request_id': 'req-1c476758-8eee-435e-a9cd-e3682f8f9f6b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:44.981900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1c476758-8eee-435e-a9cd-e3682f8f9f6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0fbadb872e74c57813900e5ebb6ac88', 'request_id': 'req-1c476758-8eee-435e-a9cd-e3682f8f9f6b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:45.083127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1c476758-8eee-435e-a9cd-e3682f8f9f6b admin admin] The token cache is being invalidate because project c0fbadb872e74c57813900e5ebb6ac88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:31:45.083413 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1c476758-8eee-435e-a9cd-e3682f8f9f6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:45.084117 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1c476758-8eee-435e-a9cd-e3682f8f9f6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:45.085995 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1299/2569] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:31:44 2022] DELETE /identity/v3/projects/c0fbadb872e74c57813900e5ebb6ac88 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:45.091082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e9e72-0b11-4165-aaef-5b7e9e10cc29 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:45.091360 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e9e72-0b11-4165-aaef-5b7e9e10cc29 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:45.091628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e9e72-0b11-4165-aaef-5b7e9e10cc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:45.110397 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-2f2e9e72-0b11-4165-aaef-5b7e9e10cc29 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:45.111655 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2f2e9e72-0b11-4165-aaef-5b7e9e10cc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:45.171282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1271/2570] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:45.179838 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed01372-a62a-4ff7-85e0-d242defabe0c None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:45.180120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed01372-a62a-4ff7-85e0-d242defabe0c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:45.180392 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed01372-a62a-4ff7-85e0-d242defabe0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:45.199717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-4ed01372-a62a-4ff7-85e0-d242defabe0c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:45.200955 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4ed01372-a62a-4ff7-85e0-d242defabe0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:45.208780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1300/2571] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:45.213179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e9e72-0b11-4165-aaef-5b7e9e10cc29 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:45.223436 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e9e72-0b11-4165-aaef-5b7e9e10cc29 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:45.228460 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffda65-aeaa-461d-8d20-f5a85a2efeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:45.229099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffda65-aeaa-461d-8d20-f5a85a2efeed admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:45.229384 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffda65-aeaa-461d-8d20-f5a85a2efeed admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:45.229656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffda65-aeaa-461d-8d20-f5a85a2efeed admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:45.230315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffda65-aeaa-461d-8d20-f5a85a2efeed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:31:45.230618 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffda65-aeaa-461d-8d20-f5a85a2efeed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:45.232244 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffda65-aeaa-461d-8d20-f5a85a2efeed admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:45.245034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1272/2572] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:31:45.251913 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed01372-a62a-4ff7-85e0-d242defabe0c None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:45.262759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed01372-a62a-4ff7-85e0-d242defabe0c None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:45.269025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98fbeb-4ea2-4314-990e-ab862c99b25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:45.269659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98fbeb-4ea2-4314-990e-ab862c99b25e admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:45.270019 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98fbeb-4ea2-4314-990e-ab862c99b25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:45.270298 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98fbeb-4ea2-4314-990e-ab862c99b25e admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:45.272210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98fbeb-4ea2-4314-990e-ab862c99b25e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:45.273055 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98fbeb-4ea2-4314-990e-ab862c99b25e admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:45.421774 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:45.431732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1301/2573] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:31:45 2022] POST /identity/v3/projects => generated 355 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:31:45.435907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffda65-aeaa-461d-8d20-f5a85a2efeed admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:45.447184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffda65-aeaa-461d-8d20-f5a85a2efeed admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:45.490004 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:45.513731 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5a482-fc4d-4595-acd1-8ed86f229a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:45.515904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5a482-fc4d-4595-acd1-8ed86f229a0c admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:45.515904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5a482-fc4d-4595-acd1-8ed86f229a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:45.515904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5a482-fc4d-4595-acd1-8ed86f229a0c admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:45.515904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5a482-fc4d-4595-acd1-8ed86f229a0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:45.517570 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5a482-fc4d-4595-acd1-8ed86f229a0c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:45.626831 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1273/2574] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:31:45 2022] POST /identity/v3/users => generated 278 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:31:45.631363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98fbeb-4ea2-4314-990e-ab862c99b25e admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:45.641481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98fbeb-4ea2-4314-990e-ab862c99b25e admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:45.646143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b6d9d-20a2-43d3-a851-05f92d13fd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:45.646731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b6d9d-20a2-43d3-a851-05f92d13fd2d admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:45.647018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b6d9d-20a2-43d3-a851-05f92d13fd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:45.647292 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b6d9d-20a2-43d3-a851-05f92d13fd2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:45.647932 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b6d9d-20a2-43d3-a851-05f92d13fd2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:45.649553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b6d9d-20a2-43d3-a851-05f92d13fd2d admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:45.659316 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1302/2575] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:31:45.663320 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5a482-fc4d-4595-acd1-8ed86f229a0c admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:45.674147 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5a482-fc4d-4595-acd1-8ed86f229a0c admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:45.679595 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9250009-7128-4981-9b8c-189ad65ad7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:45.680265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9250009-7128-4981-9b8c-189ad65ad7fe admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:45.680613 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9250009-7128-4981-9b8c-189ad65ad7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:45.680952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9250009-7128-4981-9b8c-189ad65ad7fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:45.681597 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9250009-7128-4981-9b8c-189ad65ad7fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:45.683399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9250009-7128-4981-9b8c-189ad65ad7fe admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:45.695046 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1274/2576] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:31:45.699608 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b6d9d-20a2-43d3-a851-05f92d13fd2d admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:45.709936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b6d9d-20a2-43d3-a851-05f92d13fd2d admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:45.715255 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af736b-2b3b-4460-a851-3764c21acebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:45.716195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af736b-2b3b-4460-a851-3764c21acebc admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:45.716544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af736b-2b3b-4460-a851-3764c21acebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:45.716915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af736b-2b3b-4460-a851-3764c21acebc admin admin] PATH_INFO: `/v3/projects/22173887a0b44ba79d83bbe8d786e411/users/17cdefa08847443697e485f11c3c7df0/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:45.734924 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af736b-2b3b-4460-a851-3764c21acebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=22173887a0b44ba79d83bbe8d786e411, user_id=17cdefa08847443697e485f11c3c7df0, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:45.734924 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af736b-2b3b-4460-a851-3764c21acebc admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:45.870773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1303/2577] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:45 2022] PUT /identity/v3/projects/22173887a0b44ba79d83bbe8d786e411/users/17cdefa08847443697e485f11c3c7df0/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:45.883953 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcf53b-0dad-43fe-9c71-025790c7645d None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:45.884250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcf53b-0dad-43fe-9c71-025790c7645d None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:45.884527 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcf53b-0dad-43fe-9c71-025790c7645d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:45.913064 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-59dcf53b-0dad-43fe-9c71-025790c7645d None None] MFA Rules not processed for user `17cdefa08847443697e485f11c3c7df0`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:45.914349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-59dcf53b-0dad-43fe-9c71-025790c7645d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:45.979637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1275/2578] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:45.984335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af736b-2b3b-4460-a851-3764c21acebc admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:46.048318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af736b-2b3b-4460-a851-3764c21acebc admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:46.053316 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4567fe-650e-4865-8ea7-a518fdd5087c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:46.053903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4567fe-650e-4865-8ea7-a518fdd5087c admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:46.054188 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4567fe-650e-4865-8ea7-a518fdd5087c admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:46.054466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4567fe-650e-4865-8ea7-a518fdd5087c admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:46.055101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4567fe-650e-4865-8ea7-a518fdd5087c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:46.057088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4567fe-650e-4865-8ea7-a518fdd5087c admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:46.136371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1304/2579] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:31:45 2022] POST /identity/v3/projects => generated 355 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:31:46.138952 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcf53b-0dad-43fe-9c71-025790c7645d None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:46.150944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcf53b-0dad-43fe-9c71-025790c7645d None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:46.212133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9526eb-f2d1-4f5a-a0c7-5c624a6f653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:46.212815 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9526eb-f2d1-4f5a-a0c7-5c624a6f653a admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:46.213071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9526eb-f2d1-4f5a-a0c7-5c624a6f653a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:46.213346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9526eb-f2d1-4f5a-a0c7-5c624a6f653a admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:46.213979 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9526eb-f2d1-4f5a-a0c7-5c624a6f653a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:46.215987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9526eb-f2d1-4f5a-a0c7-5c624a6f653a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:46.286072 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1276/2580] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:31:46 2022] POST /identity/v3/users => generated 284 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:31:46.287359 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4567fe-650e-4865-8ea7-a518fdd5087c admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:46.297623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4567fe-650e-4865-8ea7-a518fdd5087c admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:46.303003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfba4c9-57c7-4d26-8c6c-4c29d8a252f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:46.303586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfba4c9-57c7-4d26-8c6c-4c29d8a252f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:46.303867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfba4c9-57c7-4d26-8c6c-4c29d8a252f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:46.304450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfba4c9-57c7-4d26-8c6c-4c29d8a252f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:46.304912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfba4c9-57c7-4d26-8c6c-4c29d8a252f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:46.306500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfba4c9-57c7-4d26-8c6c-4c29d8a252f9 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:46.317514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1305/2581] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:31:46.321780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9526eb-f2d1-4f5a-a0c7-5c624a6f653a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:46.332488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9526eb-f2d1-4f5a-a0c7-5c624a6f653a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:46.337707 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc28054-00ae-47cd-99bf-4261e2d2d43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:46.338350 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc28054-00ae-47cd-99bf-4261e2d2d43f admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:46.338686 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc28054-00ae-47cd-99bf-4261e2d2d43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:46.339013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc28054-00ae-47cd-99bf-4261e2d2d43f admin admin] PATH_INFO: `/v3/projects/0d6bf03ea2ff471f8ae61b1bd3a3eb4c/users/dffda2559a4343079e1e1e8c77f87f05/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:46.354385 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc28054-00ae-47cd-99bf-4261e2d2d43f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6bf03ea2ff471f8ae61b1bd3a3eb4c, user_id=dffda2559a4343079e1e1e8c77f87f05, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:46.357197 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc28054-00ae-47cd-99bf-4261e2d2d43f admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:46.460940 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1277/2582] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:46 2022] PUT /identity/v3/projects/0d6bf03ea2ff471f8ae61b1bd3a3eb4c/users/dffda2559a4343079e1e1e8c77f87f05/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:31:46.464958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfba4c9-57c7-4d26-8c6c-4c29d8a252f9 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:46.474988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfba4c9-57c7-4d26-8c6c-4c29d8a252f9 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:46.530931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a00d1-2e8d-454c-81e2-16db99fa459f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:46.531552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a00d1-2e8d-454c-81e2-16db99fa459f admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:46.531835 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a00d1-2e8d-454c-81e2-16db99fa459f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:46.532106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a00d1-2e8d-454c-81e2-16db99fa459f admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:46.532802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a00d1-2e8d-454c-81e2-16db99fa459f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:46.534380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a00d1-2e8d-454c-81e2-16db99fa459f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:46.544298 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1306/2583] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:31:46.548438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc28054-00ae-47cd-99bf-4261e2d2d43f admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:46.559178 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc28054-00ae-47cd-99bf-4261e2d2d43f admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:46.563880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a6b9c-b00a-4f40-ab87-501af52d11c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:46.564517 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a6b9c-b00a-4f40-ab87-501af52d11c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:46.564836 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a6b9c-b00a-4f40-ab87-501af52d11c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:46.565123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a6b9c-b00a-4f40-ab87-501af52d11c5 admin admin] PATH_INFO: `/v3/projects/0d6bf03ea2ff471f8ae61b1bd3a3eb4c/users/dffda2559a4343079e1e1e8c77f87f05/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:46.567659 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a6b9c-b00a-4f40-ab87-501af52d11c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6bf03ea2ff471f8ae61b1bd3a3eb4c, user_id=dffda2559a4343079e1e1e8c77f87f05, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:46.570333 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a6b9c-b00a-4f40-ab87-501af52d11c5 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:46.572499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:46.580778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1278/2584] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:46 2022] PUT /identity/v3/projects/0d6bf03ea2ff471f8ae61b1bd3a3eb4c/users/dffda2559a4343079e1e1e8c77f87f05/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:46.591069 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a7e4e-a4bd-4689-9682-447a81bb0547 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:46.591355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a7e4e-a4bd-4689-9682-447a81bb0547 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:46.591631 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a7e4e-a4bd-4689-9682-447a81bb0547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:46.619869 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-ce0a7e4e-a4bd-4689-9682-447a81bb0547 None None] MFA Rules not processed for user `dffda2559a4343079e1e1e8c77f87f05`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:46.621172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ce0a7e4e-a4bd-4689-9682-447a81bb0547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:46.641922 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:46.690833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1307/2585] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:46.702644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a6b9c-b00a-4f40-ab87-501af52d11c5 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:46.704422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a15a6b9c-b00a-4f40-ab87-501af52d11c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:46.762907 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a6b9c-b00a-4f40-ab87-501af52d11c5 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:46.766544 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12ed19-97b8-4b67-a338-05c4c180753f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:46.767140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12ed19-97b8-4b67-a338-05c4c180753f service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:46.767422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12ed19-97b8-4b67-a338-05c4c180753f service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:46.767700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12ed19-97b8-4b67-a338-05c4c180753f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:46.776449 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12ed19-97b8-4b67-a338-05c4c180753f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:46.777910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12ed19-97b8-4b67-a338-05c4c180753f service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:46.787239 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1279/2586] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:31:46 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:31:47.001859 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a7e4e-a4bd-4689-9682-447a81bb0547 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:47.003323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ce0a7e4e-a4bd-4689-9682-447a81bb0547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:47.016480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a7e4e-a4bd-4689-9682-447a81bb0547 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:47.020427 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4d29a-e419-445c-b588-e2d2e0056ccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:47.028553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4d29a-e419-445c-b588-e2d2e0056ccf service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:47.028553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4d29a-e419-445c-b588-e2d2e0056ccf service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:47.028553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4d29a-e419-445c-b588-e2d2e0056ccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:47.083077 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4d29a-e419-445c-b588-e2d2e0056ccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:47.084536 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4d29a-e419-445c-b588-e2d2e0056ccf service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:47.094109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1308/2587] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:31:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:31:52.031383 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12ed19-97b8-4b67-a338-05c4c180753f service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:52.032883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4e12ed19-97b8-4b67-a338-05c4c180753f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:52.108187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12ed19-97b8-4b67-a338-05c4c180753f service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:52.111258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeccbe6-ed8f-4ab2-ba64-33b9ea87bf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:52.111916 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeccbe6-ed8f-4ab2-ba64-33b9ea87bf8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:52.112216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeccbe6-ed8f-4ab2-ba64-33b9ea87bf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:52.112489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeccbe6-ed8f-4ab2-ba64-33b9ea87bf8c admin admin] PATH_INFO: `/v3/users/ac1814a155c643e68f62b33b97cbda22` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:52.113729 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeccbe6-ed8f-4ab2-ba64-33b9ea87bf8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1814a155c643e68f62b33b97cbda22)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:52.115640 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeccbe6-ed8f-4ab2-ba64-33b9ea87bf8c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:52.331496 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-caeccbe6-ed8f-4ab2-ba64-33b9ea87bf8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1814a155c643e68f62b33b97cbda22'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:52.332085 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-caeccbe6-ed8f-4ab2-ba64-33b9ea87bf8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1814a155c643e68f62b33b97cbda22'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:52.420510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-caeccbe6-ed8f-4ab2-ba64-33b9ea87bf8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1814a155c643e68f62b33b97cbda22'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:52.433174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-caeccbe6-ed8f-4ab2-ba64-33b9ea87bf8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1814a155c643e68f62b33b97cbda22'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:52.440540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-caeccbe6-ed8f-4ab2-ba64-33b9ea87bf8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1814a155c643e68f62b33b97cbda22'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:52.446371 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1280/2588] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:31:52 2022] DELETE /identity/v3/users/ac1814a155c643e68f62b33b97cbda22 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:52.451093 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776de59d-b6a6-4e7f-b135-b0eaf065ffed None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:52.451562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776de59d-b6a6-4e7f-b135-b0eaf065ffed None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:52.451659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776de59d-b6a6-4e7f-b135-b0eaf065ffed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:52.471464 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-776de59d-b6a6-4e7f-b135-b0eaf065ffed None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:52.472716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-776de59d-b6a6-4e7f-b135-b0eaf065ffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:52.533009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1309/2589] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:31:52.542681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeccbe6-ed8f-4ab2-ba64-33b9ea87bf8c admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:52.546379 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-caeccbe6-ed8f-4ab2-ba64-33b9ea87bf8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:52.565391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:52.603840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeccbe6-ed8f-4ab2-ba64-33b9ea87bf8c admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:52.607489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4150115-47d5-428f-bc26-ab09b2fa81c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:52.608138 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4150115-47d5-428f-bc26-ab09b2fa81c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:52.614627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4150115-47d5-428f-bc26-ab09b2fa81c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:52.614627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4150115-47d5-428f-bc26-ab09b2fa81c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:52.618560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4150115-47d5-428f-bc26-ab09b2fa81c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:52.620019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4150115-47d5-428f-bc26-ab09b2fa81c8 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:52.629932 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1281/2590] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:31:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:31:53.166394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776de59d-b6a6-4e7f-b135-b0eaf065ffed None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:53.168208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-776de59d-b6a6-4e7f-b135-b0eaf065ffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:53.188902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776de59d-b6a6-4e7f-b135-b0eaf065ffed None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:53.198902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:53.198902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:53.198902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:53.198902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88 admin admin] PATH_INFO: `/v3/projects/d803006ec9a240be8c4c43d7d3843ca3` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:53.199425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d803006ec9a240be8c4c43d7d3843ca3)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:53.199425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:53.279262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4150115-47d5-428f-bc26-ab09b2fa81c8 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:53.279262 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c4150115-47d5-428f-bc26-ab09b2fa81c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:53.293461 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4150115-47d5-428f-bc26-ab09b2fa81c8 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:53.297387 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd95e11-541e-4cf6-9969-99da9c60fc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:53.298139 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd95e11-541e-4cf6-9969-99da9c60fc81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:53.298424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd95e11-541e-4cf6-9969-99da9c60fc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:53.298701 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd95e11-541e-4cf6-9969-99da9c60fc81 admin admin] PATH_INFO: `/v3/users/a1224f280408487391fbe28d50a789c7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:53.299871 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd95e11-541e-4cf6-9969-99da9c60fc81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1224f280408487391fbe28d50a789c7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:53.302527 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd95e11-541e-4cf6-9969-99da9c60fc81 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:53.373217 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd803006ec9a240be8c4c43d7d3843ca3', 'request_id': 'req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:53.375178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:53.379074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd803006ec9a240be8c4c43d7d3843ca3', 'request_id': 'req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:53.385409 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd803006ec9a240be8c4c43d7d3843ca3', 'request_id': 'req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:53.502214 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88 admin admin] The token cache is being invalidate because project d803006ec9a240be8c4c43d7d3843ca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:31:53.502954 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:53.503124 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:53.504756 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1310/2591] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:31:53 2022] DELETE /identity/v3/projects/d803006ec9a240be8c4c43d7d3843ca3 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:53.508533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:53.509906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:53.572865 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9f9d4-806e-4d2a-87ba-307cd3717b88 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:53.577717 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-5cd95e11-541e-4cf6-9969-99da9c60fc81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1224f280408487391fbe28d50a789c7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:53.578058 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca3a8d-bfc1-4163-84e8-c004e2b8fbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:53.578359 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-5cd95e11-541e-4cf6-9969-99da9c60fc81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1224f280408487391fbe28d50a789c7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:53.578700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca3a8d-bfc1-4163-84e8-c004e2b8fbf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:53.579001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca3a8d-bfc1-4163-84e8-c004e2b8fbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:53.579379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca3a8d-bfc1-4163-84e8-c004e2b8fbf5 admin admin] PATH_INFO: `/v3/users/174ee9a40726451ab2a4fe24b9f873ef` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:53.581180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca3a8d-bfc1-4163-84e8-c004e2b8fbf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=174ee9a40726451ab2a4fe24b9f873ef)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:53.587002 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca3a8d-bfc1-4163-84e8-c004e2b8fbf5 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:53.666135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-5cd95e11-541e-4cf6-9969-99da9c60fc81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1224f280408487391fbe28d50a789c7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:53.679086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-5cd95e11-541e-4cf6-9969-99da9c60fc81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1224f280408487391fbe28d50a789c7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:53.686128 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-5cd95e11-541e-4cf6-9969-99da9c60fc81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1224f280408487391fbe28d50a789c7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:53.692008 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1282/2592] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:31:53 2022] DELETE /identity/v3/users/a1224f280408487391fbe28d50a789c7 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:31:53.696613 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd88845-0679-4f9b-8826-c7918fdd836d None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:53.696935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd88845-0679-4f9b-8826-c7918fdd836d None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:53.697217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd88845-0679-4f9b-8826-c7918fdd836d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:53.717678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-4cd88845-0679-4f9b-8826-c7918fdd836d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:53.719355 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4cd88845-0679-4f9b-8826-c7918fdd836d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:53.750685 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-daca3a8d-bfc1-4163-84e8-c004e2b8fbf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '174ee9a40726451ab2a4fe24b9f873ef'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:53.779403 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1283/2593] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:31:53.789247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd88845-0679-4f9b-8826-c7918fdd836d None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:53.790623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4cd88845-0679-4f9b-8826-c7918fdd836d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:53.824740 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:53.848868 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd88845-0679-4f9b-8826-c7918fdd836d None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:53.852491 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea84995-c612-4780-997b-6e0b07661254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:53.853130 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea84995-c612-4780-997b-6e0b07661254 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:53.853438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea84995-c612-4780-997b-6e0b07661254 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:53.853714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea84995-c612-4780-997b-6e0b07661254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:53.863048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea84995-c612-4780-997b-6e0b07661254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:53.865371 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea84995-c612-4780-997b-6e0b07661254 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:53.874266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1284/2594] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:31:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:31:53.895042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-daca3a8d-bfc1-4163-84e8-c004e2b8fbf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '174ee9a40726451ab2a4fe24b9f873ef'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:53.895740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-daca3a8d-bfc1-4163-84e8-c004e2b8fbf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '174ee9a40726451ab2a4fe24b9f873ef'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:53.904966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-daca3a8d-bfc1-4163-84e8-c004e2b8fbf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '174ee9a40726451ab2a4fe24b9f873ef'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:53.917250 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-daca3a8d-bfc1-4163-84e8-c004e2b8fbf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '174ee9a40726451ab2a4fe24b9f873ef'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:53.923681 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1311/2595] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:31:53 2022] DELETE /identity/v3/users/174ee9a40726451ab2a4fe24b9f873ef => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:31:54.422640 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea84995-c612-4780-997b-6e0b07661254 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:54.437207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4ea84995-c612-4780-997b-6e0b07661254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:54.437732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca3a8d-bfc1-4163-84e8-c004e2b8fbf5 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:54.437732 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-daca3a8d-bfc1-4163-84e8-c004e2b8fbf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:54.440158 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea84995-c612-4780-997b-6e0b07661254 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:54.505654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca3a8d-bfc1-4163-84e8-c004e2b8fbf5 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:54.511065 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce382336-276e-45c0-9c48-58d50f5580bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:54.511694 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce382336-276e-45c0-9c48-58d50f5580bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:54.511974 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce382336-276e-45c0-9c48-58d50f5580bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:54.512251 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce382336-276e-45c0-9c48-58d50f5580bf admin admin] PATH_INFO: `/v3/projects/3666c83d26314e4d885a782a0d473a80` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:54.513586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce382336-276e-45c0-9c48-58d50f5580bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3666c83d26314e4d885a782a0d473a80)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:54.516627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce382336-276e-45c0-9c48-58d50f5580bf admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:54.518278 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e8c5a-275b-4f7c-aedd-a07f01874ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:54.518459 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:54.518904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e8c5a-275b-4f7c-aedd-a07f01874ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:54.519185 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e8c5a-275b-4f7c-aedd-a07f01874ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:54.519542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e8c5a-275b-4f7c-aedd-a07f01874ba7 admin admin] PATH_INFO: `/v3/projects/eff31dd0e51d423587ae9a49457bfae5` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:54.520709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e8c5a-275b-4f7c-aedd-a07f01874ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eff31dd0e51d423587ae9a49457bfae5)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:54.522671 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e8c5a-275b-4f7c-aedd-a07f01874ba7 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:54.708492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ce382336-276e-45c0-9c48-58d50f5580bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3666c83d26314e4d885a782a0d473a80', 'request_id': 'req-ce382336-276e-45c0-9c48-58d50f5580bf'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:54.716171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ce382336-276e-45c0-9c48-58d50f5580bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3666c83d26314e4d885a782a0d473a80', 'request_id': 'req-ce382336-276e-45c0-9c48-58d50f5580bf'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:54.723538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ce382336-276e-45c0-9c48-58d50f5580bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3666c83d26314e4d885a782a0d473a80', 'request_id': 'req-ce382336-276e-45c0-9c48-58d50f5580bf'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:54.738703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-999e8c5a-275b-4f7c-aedd-a07f01874ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eff31dd0e51d423587ae9a49457bfae5', 'request_id': 'req-999e8c5a-275b-4f7c-aedd-a07f01874ba7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:54.745797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-999e8c5a-275b-4f7c-aedd-a07f01874ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eff31dd0e51d423587ae9a49457bfae5', 'request_id': 'req-999e8c5a-275b-4f7c-aedd-a07f01874ba7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:54.752603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-999e8c5a-275b-4f7c-aedd-a07f01874ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eff31dd0e51d423587ae9a49457bfae5', 'request_id': 'req-999e8c5a-275b-4f7c-aedd-a07f01874ba7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:54.883425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ce382336-276e-45c0-9c48-58d50f5580bf admin admin] The token cache is being invalidate because project 3666c83d26314e4d885a782a0d473a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:31:54.883721 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ce382336-276e-45c0-9c48-58d50f5580bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:54.884361 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ce382336-276e-45c0-9c48-58d50f5580bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:54.885870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1312/2596] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:31:54 2022] DELETE /identity/v3/projects/3666c83d26314e4d885a782a0d473a80 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:54.890691 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324a124-42ce-47d7-93f5-cd0e21b7d317 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:54.891088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324a124-42ce-47d7-93f5-cd0e21b7d317 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:54.891459 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324a124-42ce-47d7-93f5-cd0e21b7d317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:54.911133 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-2324a124-42ce-47d7-93f5-cd0e21b7d317 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:54.912335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2324a124-42ce-47d7-93f5-cd0e21b7d317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:54.974353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1313/2597] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:31:54.986525 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbb588-0004-400e-8af2-1f336d86cd5b None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:54.986816 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbb588-0004-400e-8af2-1f336d86cd5b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:54.987091 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbb588-0004-400e-8af2-1f336d86cd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.006058 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-87fbb588-0004-400e-8af2-1f336d86cd5b None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:55.007246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-87fbb588-0004-400e-8af2-1f336d86cd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:55.008927 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-999e8c5a-275b-4f7c-aedd-a07f01874ba7 admin admin] The token cache is being invalidate because project eff31dd0e51d423587ae9a49457bfae5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:31:55.009220 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-999e8c5a-275b-4f7c-aedd-a07f01874ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:55.009838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-999e8c5a-275b-4f7c-aedd-a07f01874ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:55.011326 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1285/2598] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:31:54 2022] DELETE /identity/v3/projects/eff31dd0e51d423587ae9a49457bfae5 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:55.015408 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1314/2599] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:31:55.016141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df433ff-f44c-4cf6-98c5-855e1fabcfa7 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.016423 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df433ff-f44c-4cf6-98c5-855e1fabcfa7 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.016740 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df433ff-f44c-4cf6-98c5-855e1fabcfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.019277 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbb588-0004-400e-8af2-1f336d86cd5b None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:55.029280 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbb588-0004-400e-8af2-1f336d86cd5b None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:55.034051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ed8b0-1e70-4262-9486-66f48e878854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:55.034638 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ed8b0-1e70-4262-9486-66f48e878854 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.034918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ed8b0-1e70-4262-9486-66f48e878854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.035191 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ed8b0-1e70-4262-9486-66f48e878854 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.035833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ed8b0-1e70-4262-9486-66f48e878854 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:31:55.036160 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ed8b0-1e70-4262-9486-66f48e878854 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:55.037304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-0df433ff-f44c-4cf6-98c5-855e1fabcfa7 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:55.037897 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ed8b0-1e70-4262-9486-66f48e878854 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:55.038530 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0df433ff-f44c-4cf6-98c5-855e1fabcfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:55.047857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1286/2600] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:31:55.050073 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1315/2601] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:31:55.057275 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ed8b0-1e70-4262-9486-66f48e878854 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:55.057727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3b2fd-8075-4945-92e1-692737a9e2cb None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.058019 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3b2fd-8075-4945-92e1-692737a9e2cb None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.058298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3b2fd-8075-4945-92e1-692737a9e2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.067198 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ed8b0-1e70-4262-9486-66f48e878854 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:55.071824 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba02fb-e92f-42e2-87ee-f0ab46f0ddd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:55.072519 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba02fb-e92f-42e2-87ee-f0ab46f0ddd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.072746 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba02fb-e92f-42e2-87ee-f0ab46f0ddd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.079356 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba02fb-e92f-42e2-87ee-f0ab46f0ddd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.079356 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bba02fb-e92f-42e2-87ee-f0ab46f0ddd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:55.079356 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bba02fb-e92f-42e2-87ee-f0ab46f0ddd0 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:55.079659 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-9de3b2fd-8075-4945-92e1-692737a9e2cb None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:55.080452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9de3b2fd-8075-4945-92e1-692737a9e2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:55.089082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1287/2602] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:31:55.093959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3b2fd-8075-4945-92e1-692737a9e2cb None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:55.103021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3b2fd-8075-4945-92e1-692737a9e2cb None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:55.108184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c17e5-6dbd-48a4-b017-6a43e932fc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:55.108836 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c17e5-6dbd-48a4-b017-6a43e932fc91 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.109115 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c17e5-6dbd-48a4-b017-6a43e932fc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.109384 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c17e5-6dbd-48a4-b017-6a43e932fc91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.110026 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c17e5-6dbd-48a4-b017-6a43e932fc91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:31:55.110327 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c17e5-6dbd-48a4-b017-6a43e932fc91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:55.111908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c17e5-6dbd-48a4-b017-6a43e932fc91 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:55.123745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1288/2603] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:31:55.127971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c17e5-6dbd-48a4-b017-6a43e932fc91 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:55.138015 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c17e5-6dbd-48a4-b017-6a43e932fc91 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:55.142754 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aed209-8c5a-431d-98f3-2f400190fcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:55.148259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aed209-8c5a-431d-98f3-2f400190fcf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.148259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aed209-8c5a-431d-98f3-2f400190fcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.148259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aed209-8c5a-431d-98f3-2f400190fcf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.148259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aed209-8c5a-431d-98f3-2f400190fcf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:55.148259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aed209-8c5a-431d-98f3-2f400190fcf5 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:55.176764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1316/2604] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:31:55 2022] POST /identity/v3/projects => generated 405 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:31:55.176764 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba02fb-e92f-42e2-87ee-f0ab46f0ddd0 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:55.222850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:55.235504 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba02fb-e92f-42e2-87ee-f0ab46f0ddd0 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:55.240194 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4da45b-8eef-4eb4-b844-28c6acd3d5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:55.240833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4da45b-8eef-4eb4-b844-28c6acd3d5c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.241114 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4da45b-8eef-4eb4-b844-28c6acd3d5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.241387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4da45b-8eef-4eb4-b844-28c6acd3d5c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.241998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4da45b-8eef-4eb4-b844-28c6acd3d5c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:55.243905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4da45b-8eef-4eb4-b844-28c6acd3d5c1 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:55.257573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1289/2605] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:31:55 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:31:55.261644 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aed209-8c5a-431d-98f3-2f400190fcf5 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:55.279906 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:55.322216 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aed209-8c5a-431d-98f3-2f400190fcf5 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:55.327528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca23d3f-c094-4155-baf3-1398b4521b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:55.328111 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca23d3f-c094-4155-baf3-1398b4521b34 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.328395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca23d3f-c094-4155-baf3-1398b4521b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.328698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca23d3f-c094-4155-baf3-1398b4521b34 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.329322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca23d3f-c094-4155-baf3-1398b4521b34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:55.331242 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca23d3f-c094-4155-baf3-1398b4521b34 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:55.350498 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1317/2606] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:31:55 2022] POST /identity/v3/users => generated 303 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:31:55.351526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4da45b-8eef-4eb4-b844-28c6acd3d5c1 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:55.361704 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4da45b-8eef-4eb4-b844-28c6acd3d5c1 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:55.366471 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb032e38-5fab-47ce-913c-1682ca5fea09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:55.367077 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb032e38-5fab-47ce-913c-1682ca5fea09 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.367363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb032e38-5fab-47ce-913c-1682ca5fea09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.367643 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb032e38-5fab-47ce-913c-1682ca5fea09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.368284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb032e38-5fab-47ce-913c-1682ca5fea09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:55.369940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb032e38-5fab-47ce-913c-1682ca5fea09 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:55.379704 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1318/2607] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:31:55.386756 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb032e38-5fab-47ce-913c-1682ca5fea09 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:55.396967 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb032e38-5fab-47ce-913c-1682ca5fea09 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:55.401585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8a31c-2065-440b-9cd1-e9f7c54f4676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:55.402155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8a31c-2065-440b-9cd1-e9f7c54f4676 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.402439 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8a31c-2065-440b-9cd1-e9f7c54f4676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.402716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8a31c-2065-440b-9cd1-e9f7c54f4676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.403918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8a31c-2065-440b-9cd1-e9f7c54f4676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:55.405501 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8a31c-2065-440b-9cd1-e9f7c54f4676 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:55.415870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1319/2608] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:31:55.421182 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8a31c-2065-440b-9cd1-e9f7c54f4676 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:55.430760 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8a31c-2065-440b-9cd1-e9f7c54f4676 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:55.435370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880efbd0-fcbf-4dd9-97e8-5e06c9fb73d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:55.435994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880efbd0-fcbf-4dd9-97e8-5e06c9fb73d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.436276 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880efbd0-fcbf-4dd9-97e8-5e06c9fb73d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.436582 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880efbd0-fcbf-4dd9-97e8-5e06c9fb73d0 admin admin] PATH_INFO: `/v3/projects/ecbfbaec69bb4d86941bf88da50e4fca/users/34043e6db02b45adaa49f0392214b0f8/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.453027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1290/2609] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:31:55 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:31:55.453620 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880efbd0-fcbf-4dd9-97e8-5e06c9fb73d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecbfbaec69bb4d86941bf88da50e4fca, user_id=34043e6db02b45adaa49f0392214b0f8, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:55.454170 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca23d3f-c094-4155-baf3-1398b4521b34 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:55.455867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880efbd0-fcbf-4dd9-97e8-5e06c9fb73d0 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:55.465172 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca23d3f-c094-4155-baf3-1398b4521b34 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:55.470564 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb968d-1319-4cfa-a896-6df900dac5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:55.471149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb968d-1319-4cfa-a896-6df900dac5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.471428 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb968d-1319-4cfa-a896-6df900dac5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.471700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb968d-1319-4cfa-a896-6df900dac5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.472337 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb968d-1319-4cfa-a896-6df900dac5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:55.473946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb968d-1319-4cfa-a896-6df900dac5e6 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:55.484008 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1291/2610] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:31:55.487741 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb968d-1319-4cfa-a896-6df900dac5e6 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:55.497838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb968d-1319-4cfa-a896-6df900dac5e6 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:55.502476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad915e1-828f-463d-9c29-39a619e2d6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:55.503020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad915e1-828f-463d-9c29-39a619e2d6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.503301 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad915e1-828f-463d-9c29-39a619e2d6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.503574 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad915e1-828f-463d-9c29-39a619e2d6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.504182 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad915e1-828f-463d-9c29-39a619e2d6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:55.505857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad915e1-828f-463d-9c29-39a619e2d6d0 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:55.517913 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1292/2611] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:31:55.521895 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad915e1-828f-463d-9c29-39a619e2d6d0 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:55.531091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad915e1-828f-463d-9c29-39a619e2d6d0 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:55.535766 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223877c6-235b-443d-8f27-7da4597d4aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:55.536374 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223877c6-235b-443d-8f27-7da4597d4aa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.536695 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223877c6-235b-443d-8f27-7da4597d4aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.536969 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223877c6-235b-443d-8f27-7da4597d4aa1 admin admin] PATH_INFO: `/v3/projects/c9b8e81b96fd43a8ab5d7f767fc5e555/users/d2e9bbee790c4c0d9833c517db5d2ad1/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.551027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223877c6-235b-443d-8f27-7da4597d4aa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b8e81b96fd43a8ab5d7f767fc5e555, user_id=d2e9bbee790c4c0d9833c517db5d2ad1, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:55.553650 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223877c6-235b-443d-8f27-7da4597d4aa1 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:55.558741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1320/2612] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:55 2022] PUT /identity/v3/projects/ecbfbaec69bb4d86941bf88da50e4fca/users/34043e6db02b45adaa49f0392214b0f8/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:31:55.564436 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03cd287-9de0-4331-974a-7141021ffdc1 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.564760 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03cd287-9de0-4331-974a-7141021ffdc1 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.565039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03cd287-9de0-4331-974a-7141021ffdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.656200 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1293/2613] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:55 2022] PUT /identity/v3/projects/c9b8e81b96fd43a8ab5d7f767fc5e555/users/d2e9bbee790c4c0d9833c517db5d2ad1/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:31:55.668372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bc6c5-7f1f-417e-98b2-2738d9a174de None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.668986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bc6c5-7f1f-417e-98b2-2738d9a174de None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.669114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bc6c5-7f1f-417e-98b2-2738d9a174de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.702419 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-666bc6c5-7f1f-417e-98b2-2738d9a174de None None] MFA Rules not processed for user `d2e9bbee790c4c0d9833c517db5d2ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:55.703555 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-666bc6c5-7f1f-417e-98b2-2738d9a174de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:55.710155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-e03cd287-9de0-4331-974a-7141021ffdc1 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:55.711594 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e03cd287-9de0-4331-974a-7141021ffdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:55.772706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1294/2614] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:31:55.776818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1321/2615] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:31:55.777317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bc6c5-7f1f-417e-98b2-2738d9a174de None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:55.788121 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03cd287-9de0-4331-974a-7141021ffdc1 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:55.788403 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bc6c5-7f1f-417e-98b2-2738d9a174de None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:55.789541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e03cd287-9de0-4331-974a-7141021ffdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:55.792958 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990e585-1f06-4cf5-8318-ebdeb6cb8fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:55.793570 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990e585-1f06-4cf5-8318-ebdeb6cb8fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.793878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990e585-1f06-4cf5-8318-ebdeb6cb8fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.794222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990e585-1f06-4cf5-8318-ebdeb6cb8fa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.794856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990e585-1f06-4cf5-8318-ebdeb6cb8fa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:55.796809 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990e585-1f06-4cf5-8318-ebdeb6cb8fa8 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:55.850047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03cd287-9de0-4331-974a-7141021ffdc1 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:55.853721 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45574d56-db0c-459b-bf8e-07b8cd233867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:55.854323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45574d56-db0c-459b-bf8e-07b8cd233867 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:55.854599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45574d56-db0c-459b-bf8e-07b8cd233867 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:55.854867 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45574d56-db0c-459b-bf8e-07b8cd233867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:55.865807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45574d56-db0c-459b-bf8e-07b8cd233867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:55.867243 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45574d56-db0c-459b-bf8e-07b8cd233867 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:55.878760 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1322/2616] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:31:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:31:56.074032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1295/2617] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:31:55 2022] POST /identity/v3/projects => generated 381 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:31:56.076284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45574d56-db0c-459b-bf8e-07b8cd233867 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:56.086264 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45574d56-db0c-459b-bf8e-07b8cd233867 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:56.121480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:56.146309 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f4a18-7006-48a0-9165-62e1aaf270aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:56.146923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f4a18-7006-48a0-9165-62e1aaf270aa admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:56.147208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f4a18-7006-48a0-9165-62e1aaf270aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:56.147482 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f4a18-7006-48a0-9165-62e1aaf270aa admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:56.148123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f4a18-7006-48a0-9165-62e1aaf270aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:56.150176 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f4a18-7006-48a0-9165-62e1aaf270aa admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:56.300583 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1323/2618] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:31:56 2022] POST /identity/v3/users => generated 297 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:31:56.302145 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990e585-1f06-4cf5-8318-ebdeb6cb8fa8 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:56.312858 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990e585-1f06-4cf5-8318-ebdeb6cb8fa8 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:56.318785 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a4f9e-3fbc-4545-8f82-9fdd5e0350ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:56.319488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a4f9e-3fbc-4545-8f82-9fdd5e0350ea admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:56.319833 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a4f9e-3fbc-4545-8f82-9fdd5e0350ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:56.320398 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a4f9e-3fbc-4545-8f82-9fdd5e0350ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:56.321433 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a4f9e-3fbc-4545-8f82-9fdd5e0350ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:56.327007 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a4f9e-3fbc-4545-8f82-9fdd5e0350ea admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:56.334388 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1296/2619] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:31:56.338849 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f4a18-7006-48a0-9165-62e1aaf270aa admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:56.352183 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f4a18-7006-48a0-9165-62e1aaf270aa admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:56.358234 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40a2a1-8a55-4c68-b5d8-7e443de92485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:56.359059 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40a2a1-8a55-4c68-b5d8-7e443de92485 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:56.359465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40a2a1-8a55-4c68-b5d8-7e443de92485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:56.359821 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40a2a1-8a55-4c68-b5d8-7e443de92485 admin admin] PATH_INFO: `/v3/projects/9bd01e3634104944a2f22ddc3183ebb3/users/531555de32ed4dbaad00fb7e8ee4b901/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:56.376299 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40a2a1-8a55-4c68-b5d8-7e443de92485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd01e3634104944a2f22ddc3183ebb3, user_id=531555de32ed4dbaad00fb7e8ee4b901, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:56.379350 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40a2a1-8a55-4c68-b5d8-7e443de92485 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:56.468120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1324/2620] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:56 2022] PUT /identity/v3/projects/9bd01e3634104944a2f22ddc3183ebb3/users/531555de32ed4dbaad00fb7e8ee4b901/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:56.472146 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a4f9e-3fbc-4545-8f82-9fdd5e0350ea admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:56.483007 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a4f9e-3fbc-4545-8f82-9fdd5e0350ea admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:56.494690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:56.546031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85e855-d834-47d3-a877-5ef46a9a3be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:56.550249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85e855-d834-47d3-a877-5ef46a9a3be6 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:56.550249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85e855-d834-47d3-a877-5ef46a9a3be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:56.550249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85e855-d834-47d3-a877-5ef46a9a3be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:56.550249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85e855-d834-47d3-a877-5ef46a9a3be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:56.550249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85e855-d834-47d3-a877-5ef46a9a3be6 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:56.562816 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1297/2621] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:31:56.563976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40a2a1-8a55-4c68-b5d8-7e443de92485 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:56.579197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40a2a1-8a55-4c68-b5d8-7e443de92485 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:56.587534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957d327-9a1a-4607-b623-dbbd008ead7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:56.588234 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957d327-9a1a-4607-b623-dbbd008ead7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:56.588513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957d327-9a1a-4607-b623-dbbd008ead7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:56.588828 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957d327-9a1a-4607-b623-dbbd008ead7e admin admin] PATH_INFO: `/v3/projects/9bd01e3634104944a2f22ddc3183ebb3/users/531555de32ed4dbaad00fb7e8ee4b901/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:56.591527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957d327-9a1a-4607-b623-dbbd008ead7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd01e3634104944a2f22ddc3183ebb3, user_id=531555de32ed4dbaad00fb7e8ee4b901, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:56.594302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957d327-9a1a-4607-b623-dbbd008ead7e admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:56.609045 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1325/2622] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:56 2022] PUT /identity/v3/projects/9bd01e3634104944a2f22ddc3183ebb3/users/531555de32ed4dbaad00fb7e8ee4b901/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:56.629422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477196c-32bf-4718-b680-ed71fa223af9 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:56.629712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477196c-32bf-4718-b680-ed71fa223af9 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:56.629992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477196c-32bf-4718-b680-ed71fa223af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:56.660528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-a477196c-32bf-4718-b680-ed71fa223af9 None None] MFA Rules not processed for user `531555de32ed4dbaad00fb7e8ee4b901`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:56.661799 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a477196c-32bf-4718-b680-ed71fa223af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:56.736861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1298/2623] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:56.749480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957d327-9a1a-4607-b623-dbbd008ead7e admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:56.751323 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-8957d327-9a1a-4607-b623-dbbd008ead7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:56.823441 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957d327-9a1a-4607-b623-dbbd008ead7e admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:56.828601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4866a3-b95b-4eed-8a41-0fc986a1efa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:56.829245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4866a3-b95b-4eed-8a41-0fc986a1efa3 service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:56.829530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4866a3-b95b-4eed-8a41-0fc986a1efa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:56.829802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4866a3-b95b-4eed-8a41-0fc986a1efa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:56.840307 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4866a3-b95b-4eed-8a41-0fc986a1efa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:56.842918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4866a3-b95b-4eed-8a41-0fc986a1efa3 service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:56.852804 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1326/2624] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:31:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:31:57.464720 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ff5f5-c816-4e04-b478-785b49cbc983 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:57.465020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ff5f5-c816-4e04-b478-785b49cbc983 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:57.465326 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ff5f5-c816-4e04-b478-785b49cbc983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:57.492974 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-7a2ff5f5-c816-4e04-b478-785b49cbc983 None None] MFA Rules not processed for user `34043e6db02b45adaa49f0392214b0f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:57.494187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-7a2ff5f5-c816-4e04-b478-785b49cbc983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:57.572236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1299/2625] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:57 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:57.577833 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4866a3-b95b-4eed-8a41-0fc986a1efa3 service cinder] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:57.642200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4866a3-b95b-4eed-8a41-0fc986a1efa3 service cinder] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:57.647304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80c46e-2d95-4aa2-9bec-138c714e0558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:57.647927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80c46e-2d95-4aa2-9bec-138c714e0558 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:57.648206 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80c46e-2d95-4aa2-9bec-138c714e0558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:57.648475 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80c46e-2d95-4aa2-9bec-138c714e0558 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:57.649187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80c46e-2d95-4aa2-9bec-138c714e0558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:57.651242 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80c46e-2d95-4aa2-9bec-138c714e0558 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:57.770744 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1327/2626] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:31:57 2022] POST /identity/v3/projects => generated 403 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:31:57.775058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ff5f5-c816-4e04-b478-785b49cbc983 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:57.786426 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ff5f5-c816-4e04-b478-785b49cbc983 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:57.812929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:57.844728 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6304b-1cf3-4590-be60-d250d5b124e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:57.845327 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6304b-1cf3-4590-be60-d250d5b124e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:57.845607 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6304b-1cf3-4590-be60-d250d5b124e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:57.845878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c6304b-1cf3-4590-be60-d250d5b124e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:57.846504 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c6304b-1cf3-4590-be60-d250d5b124e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:57.848547 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c6304b-1cf3-4590-be60-d250d5b124e7 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:57.924733 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1300/2627] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:31:57 2022] POST /identity/v3/users => generated 308 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:31:57.930951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80c46e-2d95-4aa2-9bec-138c714e0558 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:57.939260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80c46e-2d95-4aa2-9bec-138c714e0558 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:57.943812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb736e5-07cd-48bc-a0c5-a098f71d56dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:57.944404 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb736e5-07cd-48bc-a0c5-a098f71d56dd admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:57.944718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb736e5-07cd-48bc-a0c5-a098f71d56dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:57.944995 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb736e5-07cd-48bc-a0c5-a098f71d56dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:57.945640 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb736e5-07cd-48bc-a0c5-a098f71d56dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:57.947229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb736e5-07cd-48bc-a0c5-a098f71d56dd admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:57.957068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1328/2628] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:31:57.961420 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6304b-1cf3-4590-be60-d250d5b124e7 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:57.971776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c6304b-1cf3-4590-be60-d250d5b124e7 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:57.976305 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7299c-643f-452f-aecc-e9289aa0756a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:57.976977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7299c-643f-452f-aecc-e9289aa0756a admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:57.977265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7299c-643f-452f-aecc-e9289aa0756a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:57.977541 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7299c-643f-452f-aecc-e9289aa0756a admin admin] PATH_INFO: `/v3/projects/579dd6bb120849d7af9e66eea809944c/users/5a5f9b0a423f4dd488baf792de53e6e4/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:57.994141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7299c-643f-452f-aecc-e9289aa0756a admin admin] RBAC: Authorizing `identity:create_grant(project_id=579dd6bb120849d7af9e66eea809944c, user_id=5a5f9b0a423f4dd488baf792de53e6e4, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:57.995829 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7299c-643f-452f-aecc-e9289aa0756a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:58.067675 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1301/2629] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:57 2022] PUT /identity/v3/projects/579dd6bb120849d7af9e66eea809944c/users/5a5f9b0a423f4dd488baf792de53e6e4/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:58.071678 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb736e5-07cd-48bc-a0c5-a098f71d56dd admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:58.079555 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7299c-643f-452f-aecc-e9289aa0756a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:58.084680 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:58.089882 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb736e5-07cd-48bc-a0c5-a098f71d56dd admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:58.152164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7299c-643f-452f-aecc-e9289aa0756a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:58.157288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c9346-ea4b-4396-9393-8d80195703f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:58.157508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c9346-ea4b-4396-9393-8d80195703f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:58.157666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c9346-ea4b-4396-9393-8d80195703f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:58.157940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c9346-ea4b-4396-9393-8d80195703f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:58.159028 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c9346-ea4b-4396-9393-8d80195703f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:58.160189 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c9346-ea4b-4396-9393-8d80195703f5 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:58.162845 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03eac8e-a582-428e-ac19-52972c322311 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531555de32ed4dbaad00fb7e8ee4b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bd01e3634104944a2f22ddc3183ebb3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:58.163012 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03eac8e-a582-428e-ac19-52972c322311 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:58.163349 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03eac8e-a582-428e-ac19-52972c322311 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:58.163707 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03eac8e-a582-428e-ac19-52972c322311 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:58.164471 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03eac8e-a582-428e-ac19-52972c322311 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:31:58.165129 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03eac8e-a582-428e-ac19-52972c322311 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:58.166670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03eac8e-a582-428e-ac19-52972c322311 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:58.171497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1329/2630] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:31:58.181302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c9346-ea4b-4396-9393-8d80195703f5 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:58.181477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1302/2631] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:31:58.184131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03eac8e-a582-428e-ac19-52972c322311 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:58.186257 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c9346-ea4b-4396-9393-8d80195703f5 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:58.191024 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911654b0-7ef1-4959-87a6-a21e8254d595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:58.191709 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911654b0-7ef1-4959-87a6-a21e8254d595 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:58.191991 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911654b0-7ef1-4959-87a6-a21e8254d595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:58.192269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911654b0-7ef1-4959-87a6-a21e8254d595 admin admin] PATH_INFO: `/v3/projects/579dd6bb120849d7af9e66eea809944c/users/5a5f9b0a423f4dd488baf792de53e6e4/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:58.194301 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911654b0-7ef1-4959-87a6-a21e8254d595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=579dd6bb120849d7af9e66eea809944c, user_id=5a5f9b0a423f4dd488baf792de53e6e4, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:58.194470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03eac8e-a582-428e-ac19-52972c322311 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:58.197085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911654b0-7ef1-4959-87a6-a21e8254d595 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:58.199403 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa80e88-5394-4046-8e13-911f15706f71 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531555de32ed4dbaad00fb7e8ee4b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bd01e3634104944a2f22ddc3183ebb3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:58.200589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa80e88-5394-4046-8e13-911f15706f71 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:58.200905 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa80e88-5394-4046-8e13-911f15706f71 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:58.201188 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa80e88-5394-4046-8e13-911f15706f71 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:58.201816 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa80e88-5394-4046-8e13-911f15706f71 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:58.203736 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa80e88-5394-4046-8e13-911f15706f71 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:58.208795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1330/2632] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:31:58 2022] PUT /identity/v3/projects/579dd6bb120849d7af9e66eea809944c/users/5a5f9b0a423f4dd488baf792de53e6e4/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:31:58.415435 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1303/2633] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:31:58 2022] POST /identity/v3/projects => generated 354 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:31:58.416675 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911654b0-7ef1-4959-87a6-a21e8254d595 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:58.480963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911654b0-7ef1-4959-87a6-a21e8254d595 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:58.542659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec044bf-e338-4319-a860-1020c6d976eb tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531555de32ed4dbaad00fb7e8ee4b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bd01e3634104944a2f22ddc3183ebb3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:58.548306 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec044bf-e338-4319-a860-1020c6d976eb tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:58.548306 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec044bf-e338-4319-a860-1020c6d976eb tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:58.548306 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec044bf-e338-4319-a860-1020c6d976eb tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:58.548306 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec044bf-e338-4319-a860-1020c6d976eb tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:31:58.548306 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec044bf-e338-4319-a860-1020c6d976eb tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:58.548853 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec044bf-e338-4319-a860-1020c6d976eb tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:58.563734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1331/2634] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:31:58.603345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa80e88-5394-4046-8e13-911f15706f71 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:58.614347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa80e88-5394-4046-8e13-911f15706f71 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:58.618890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630bfae-0116-4da0-a88d-78af015028a0 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531555de32ed4dbaad00fb7e8ee4b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bd01e3634104944a2f22ddc3183ebb3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:58.619680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630bfae-0116-4da0-a88d-78af015028a0 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:58.619959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630bfae-0116-4da0-a88d-78af015028a0 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:58.620228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630bfae-0116-4da0-a88d-78af015028a0 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] PATH_INFO: `/v3/projects/d5e45e0aa16b4cff85e3a7dccd6b8bdb` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:58.621371 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630bfae-0116-4da0-a88d-78af015028a0 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d5e45e0aa16b4cff85e3a7dccd6b8bdb)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:58.623358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630bfae-0116-4da0-a88d-78af015028a0 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:58.825428 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6630bfae-0116-4da0-a88d-78af015028a0 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e45e0aa16b4cff85e3a7dccd6b8bdb', 'request_id': 'req-6630bfae-0116-4da0-a88d-78af015028a0'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:58.830820 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6630bfae-0116-4da0-a88d-78af015028a0 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e45e0aa16b4cff85e3a7dccd6b8bdb', 'request_id': 'req-6630bfae-0116-4da0-a88d-78af015028a0'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:58.832082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:58.838340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6630bfae-0116-4da0-a88d-78af015028a0 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e45e0aa16b4cff85e3a7dccd6b8bdb', 'request_id': 'req-6630bfae-0116-4da0-a88d-78af015028a0'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:58.956545 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6630bfae-0116-4da0-a88d-78af015028a0 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] The token cache is being invalidate because project d5e45e0aa16b4cff85e3a7dccd6b8bdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:31:58.956877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6630bfae-0116-4da0-a88d-78af015028a0 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:58.957525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6630bfae-0116-4da0-a88d-78af015028a0 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:58.959009 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1304/2635] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:31:58 2022] DELETE /identity/v3/projects/d5e45e0aa16b4cff85e3a7dccd6b8bdb => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:31:59.591334 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032d808-3d1c-4288-8898-d78ea731cca2 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:59.591800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032d808-3d1c-4288-8898-d78ea731cca2 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:59.591800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032d808-3d1c-4288-8898-d78ea731cca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:59.603598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630bfae-0116-4da0-a88d-78af015028a0 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:31:59.605302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-6630bfae-0116-4da0-a88d-78af015028a0 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:59.623753 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-a032d808-3d1c-4288-8898-d78ea731cca2 None None] MFA Rules not processed for user `5a5f9b0a423f4dd488baf792de53e6e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:31:59.625162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a032d808-3d1c-4288-8898-d78ea731cca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:31:59.646624 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:31:59.677288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630bfae-0116-4da0-a88d-78af015028a0 tempest-VolumeQuotaClassesTest-1925908364 tempest-VolumeQuotaClassesTest-1925908364-project-admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:31:59.682111 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0f7dc-c42e-48de-9373-a1a21e2437c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:31:59.682734 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0f7dc-c42e-48de-9373-a1a21e2437c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:31:59.683018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0f7dc-c42e-48de-9373-a1a21e2437c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:31:59.683290 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0f7dc-c42e-48de-9373-a1a21e2437c1 admin admin] PATH_INFO: `/v3/users/d2e9bbee790c4c0d9833c517db5d2ad1` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:31:59.684655 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0f7dc-c42e-48de-9373-a1a21e2437c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e9bbee790c4c0d9833c517db5d2ad1)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:31:59.686861 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0f7dc-c42e-48de-9373-a1a21e2437c1 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:31:59.705286 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1332/2636] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:31:59 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:31:59.914694 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-39f0f7dc-c42e-48de-9373-a1a21e2437c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e9bbee790c4c0d9833c517db5d2ad1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:31:59.915347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-39f0f7dc-c42e-48de-9373-a1a21e2437c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e9bbee790c4c0d9833c517db5d2ad1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:00.043840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-39f0f7dc-c42e-48de-9373-a1a21e2437c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e9bbee790c4c0d9833c517db5d2ad1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:00.057926 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-39f0f7dc-c42e-48de-9373-a1a21e2437c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e9bbee790c4c0d9833c517db5d2ad1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:00.066620 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-39f0f7dc-c42e-48de-9373-a1a21e2437c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e9bbee790c4c0d9833c517db5d2ad1'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:00.073212 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1305/2637] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:31:59 2022] DELETE /identity/v3/users/d2e9bbee790c4c0d9833c517db5d2ad1 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:00.078224 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee940f-8ae7-4d42-ad2a-7d6e89394166 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:00.078508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee940f-8ae7-4d42-ad2a-7d6e89394166 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:00.078826 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee940f-8ae7-4d42-ad2a-7d6e89394166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:00.097659 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-21ee940f-8ae7-4d42-ad2a-7d6e89394166 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:00.098860 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-21ee940f-8ae7-4d42-ad2a-7d6e89394166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:00.158619 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1333/2638] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:00.168576 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0f7dc-c42e-48de-9373-a1a21e2437c1 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:00.170162 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-39f0f7dc-c42e-48de-9373-a1a21e2437c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:00.235279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0f7dc-c42e-48de-9373-a1a21e2437c1 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:00.238959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75de480-5b4d-42b4-824c-ee3ab45857df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:00.239570 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75de480-5b4d-42b4-824c-ee3ab45857df service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:00.239849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75de480-5b4d-42b4-824c-ee3ab45857df service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:00.240130 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75de480-5b4d-42b4-824c-ee3ab45857df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:00.251215 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75de480-5b4d-42b4-824c-ee3ab45857df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:00.252680 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75de480-5b4d-42b4-824c-ee3ab45857df service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:00.264787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1306/2639] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:32:00.808813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee940f-8ae7-4d42-ad2a-7d6e89394166 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:00.811377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-21ee940f-8ae7-4d42-ad2a-7d6e89394166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:00.825692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee940f-8ae7-4d42-ad2a-7d6e89394166 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:00.831721 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1add973-9254-4ad8-ac47-307f2b08da8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:00.832164 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1add973-9254-4ad8-ac47-307f2b08da8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:00.832453 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1add973-9254-4ad8-ac47-307f2b08da8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:00.832758 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1add973-9254-4ad8-ac47-307f2b08da8b admin admin] PATH_INFO: `/v3/projects/c9b8e81b96fd43a8ab5d7f767fc5e555` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:00.833913 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1add973-9254-4ad8-ac47-307f2b08da8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9b8e81b96fd43a8ab5d7f767fc5e555)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:00.836181 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1add973-9254-4ad8-ac47-307f2b08da8b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:00.967871 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f1add973-9254-4ad8-ac47-307f2b08da8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b8e81b96fd43a8ab5d7f767fc5e555', 'request_id': 'req-f1add973-9254-4ad8-ac47-307f2b08da8b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:00.974155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f1add973-9254-4ad8-ac47-307f2b08da8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b8e81b96fd43a8ab5d7f767fc5e555', 'request_id': 'req-f1add973-9254-4ad8-ac47-307f2b08da8b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:00.980969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f1add973-9254-4ad8-ac47-307f2b08da8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b8e81b96fd43a8ab5d7f767fc5e555', 'request_id': 'req-f1add973-9254-4ad8-ac47-307f2b08da8b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:01.176654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f1add973-9254-4ad8-ac47-307f2b08da8b admin admin] The token cache is being invalidate because project c9b8e81b96fd43a8ab5d7f767fc5e555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:32:01.177167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f1add973-9254-4ad8-ac47-307f2b08da8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:01.190674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f1add973-9254-4ad8-ac47-307f2b08da8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:01.193339 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1334/2640] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:32:00 2022] DELETE /identity/v3/projects/c9b8e81b96fd43a8ab5d7f767fc5e555 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:01.197583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75de480-5b4d-42b4-824c-ee3ab45857df service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:01.208749 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75de480-5b4d-42b4-824c-ee3ab45857df service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:01.214452 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:01.272786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3852e-ef7f-4a1d-b3e0-3cc7a8fc9188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:01.273423 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3852e-ef7f-4a1d-b3e0-3cc7a8fc9188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:01.273708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3852e-ef7f-4a1d-b3e0-3cc7a8fc9188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:01.273985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3852e-ef7f-4a1d-b3e0-3cc7a8fc9188 admin admin] PATH_INFO: `/v3/users/531555de32ed4dbaad00fb7e8ee4b901` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:01.275155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3852e-ef7f-4a1d-b3e0-3cc7a8fc9188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=531555de32ed4dbaad00fb7e8ee4b901)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:01.277171 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3852e-ef7f-4a1d-b3e0-3cc7a8fc9188 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:01.331478 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1add973-9254-4ad8-ac47-307f2b08da8b admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:01.333116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f1add973-9254-4ad8-ac47-307f2b08da8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:01.347527 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1add973-9254-4ad8-ac47-307f2b08da8b admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:01.352197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19e16d-4fff-43c1-8462-75a02855d98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:01.352917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19e16d-4fff-43c1-8462-75a02855d98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:01.353241 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19e16d-4fff-43c1-8462-75a02855d98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:01.353476 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19e16d-4fff-43c1-8462-75a02855d98a admin admin] PATH_INFO: `/v3/users/34043e6db02b45adaa49f0392214b0f8` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:01.354642 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19e16d-4fff-43c1-8462-75a02855d98a admin admin] RBAC: Authorizing `identity:delete_user(user_id=34043e6db02b45adaa49f0392214b0f8)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:01.357170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19e16d-4fff-43c1-8462-75a02855d98a admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:01.475965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-bba3852e-ef7f-4a1d-b3e0-3cc7a8fc9188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '531555de32ed4dbaad00fb7e8ee4b901'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:01.476803 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-bba3852e-ef7f-4a1d-b3e0-3cc7a8fc9188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '531555de32ed4dbaad00fb7e8ee4b901'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:01.564903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:01.574971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-bba3852e-ef7f-4a1d-b3e0-3cc7a8fc9188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '531555de32ed4dbaad00fb7e8ee4b901'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:01.580567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5a19e16d-4fff-43c1-8462-75a02855d98a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34043e6db02b45adaa49f0392214b0f8'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:01.589744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-bba3852e-ef7f-4a1d-b3e0-3cc7a8fc9188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '531555de32ed4dbaad00fb7e8ee4b901'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:01.598137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-bba3852e-ef7f-4a1d-b3e0-3cc7a8fc9188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '531555de32ed4dbaad00fb7e8ee4b901'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:01.604316 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1307/2641] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:32:01 2022] DELETE /identity/v3/users/531555de32ed4dbaad00fb7e8ee4b901 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:32:01.666451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5a19e16d-4fff-43c1-8462-75a02855d98a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34043e6db02b45adaa49f0392214b0f8'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:01.667109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5a19e16d-4fff-43c1-8462-75a02855d98a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34043e6db02b45adaa49f0392214b0f8'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:01.675289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5a19e16d-4fff-43c1-8462-75a02855d98a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34043e6db02b45adaa49f0392214b0f8'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:01.688061 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-5a19e16d-4fff-43c1-8462-75a02855d98a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34043e6db02b45adaa49f0392214b0f8'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:01.694148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1335/2642] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:32:01 2022] DELETE /identity/v3/users/34043e6db02b45adaa49f0392214b0f8 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:01.970960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3852e-ef7f-4a1d-b3e0-3cc7a8fc9188 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:01.980718 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-bba3852e-ef7f-4a1d-b3e0-3cc7a8fc9188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:02.052618 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3852e-ef7f-4a1d-b3e0-3cc7a8fc9188 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:02.091710 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:02.108983 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19e16d-4fff-43c1-8462-75a02855d98a admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:02.110685 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-5a19e16d-4fff-43c1-8462-75a02855d98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:02.116263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef3bde-80c8-40db-9716-49076f9f2e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:02.117050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef3bde-80c8-40db-9716-49076f9f2e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:02.117335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef3bde-80c8-40db-9716-49076f9f2e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:02.117615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef3bde-80c8-40db-9716-49076f9f2e20 admin admin] PATH_INFO: `/v3/projects/ecbfbaec69bb4d86941bf88da50e4fca` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:02.118746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef3bde-80c8-40db-9716-49076f9f2e20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecbfbaec69bb4d86941bf88da50e4fca)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:02.120665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef3bde-80c8-40db-9716-49076f9f2e20 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:02.130236 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19e16d-4fff-43c1-8462-75a02855d98a admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:02.135708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53215b-fa4e-4bfb-8158-593e55187331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:02.136834 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53215b-fa4e-4bfb-8158-593e55187331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:02.137710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53215b-fa4e-4bfb-8158-593e55187331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:02.137920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53215b-fa4e-4bfb-8158-593e55187331 admin admin] PATH_INFO: `/v3/projects/9bd01e3634104944a2f22ddc3183ebb3` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:02.139606 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53215b-fa4e-4bfb-8158-593e55187331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd01e3634104944a2f22ddc3183ebb3)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:02.141746 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53215b-fa4e-4bfb-8158-593e55187331 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:02.227356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-cdef3bde-80c8-40db-9716-49076f9f2e20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecbfbaec69bb4d86941bf88da50e4fca', 'request_id': 'req-cdef3bde-80c8-40db-9716-49076f9f2e20'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:02.232965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-cdef3bde-80c8-40db-9716-49076f9f2e20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecbfbaec69bb4d86941bf88da50e4fca', 'request_id': 'req-cdef3bde-80c8-40db-9716-49076f9f2e20'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:02.239122 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-cdef3bde-80c8-40db-9716-49076f9f2e20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecbfbaec69bb4d86941bf88da50e4fca', 'request_id': 'req-cdef3bde-80c8-40db-9716-49076f9f2e20'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:02.257834 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ac53215b-fa4e-4bfb-8158-593e55187331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd01e3634104944a2f22ddc3183ebb3', 'request_id': 'req-ac53215b-fa4e-4bfb-8158-593e55187331'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:02.264529 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ac53215b-fa4e-4bfb-8158-593e55187331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd01e3634104944a2f22ddc3183ebb3', 'request_id': 'req-ac53215b-fa4e-4bfb-8158-593e55187331'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:02.271148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ac53215b-fa4e-4bfb-8158-593e55187331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd01e3634104944a2f22ddc3183ebb3', 'request_id': 'req-ac53215b-fa4e-4bfb-8158-593e55187331'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:02.340046 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-cdef3bde-80c8-40db-9716-49076f9f2e20 admin admin] The token cache is being invalidate because project ecbfbaec69bb4d86941bf88da50e4fca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:32:02.340382 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-cdef3bde-80c8-40db-9716-49076f9f2e20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:02.341042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-cdef3bde-80c8-40db-9716-49076f9f2e20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:02.342523 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1308/2643] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:32:01 2022] DELETE /identity/v3/projects/ecbfbaec69bb4d86941bf88da50e4fca => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:02.346483 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef3bde-80c8-40db-9716-49076f9f2e20 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:02.347795 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-cdef3bde-80c8-40db-9716-49076f9f2e20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:02.410425 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef3bde-80c8-40db-9716-49076f9f2e20 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:02.420337 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ac53215b-fa4e-4bfb-8158-593e55187331 admin admin] The token cache is being invalidate because project 9bd01e3634104944a2f22ddc3183ebb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:32:02.424168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ac53215b-fa4e-4bfb-8158-593e55187331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:02.424168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ac53215b-fa4e-4bfb-8158-593e55187331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:02.424168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1336/2644] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:32:02 2022] DELETE /identity/v3/projects/9bd01e3634104944a2f22ddc3183ebb3 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:02.427771 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e69f2-94d1-4ed5-954a-04ccb52bec62 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:02.428085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e69f2-94d1-4ed5-954a-04ccb52bec62 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:02.428390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e69f2-94d1-4ed5-954a-04ccb52bec62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:02.448818 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-314e69f2-94d1-4ed5-954a-04ccb52bec62 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:02.450929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-314e69f2-94d1-4ed5-954a-04ccb52bec62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:02.468240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449564b-4470-4f56-b636-3709cc34dfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:02.468900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449564b-4470-4f56-b636-3709cc34dfd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:02.469183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449564b-4470-4f56-b636-3709cc34dfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:02.469454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449564b-4470-4f56-b636-3709cc34dfd5 admin admin] PATH_INFO: `/v3/users/5a5f9b0a423f4dd488baf792de53e6e4` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:02.470718 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449564b-4470-4f56-b636-3709cc34dfd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a5f9b0a423f4dd488baf792de53e6e4)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:02.472663 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449564b-4470-4f56-b636-3709cc34dfd5 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:02.510329 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1337/2645] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:32:02.518467 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865af89-89c8-4b56-9131-56abaa90694f None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:02.518782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865af89-89c8-4b56-9131-56abaa90694f None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:02.519064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865af89-89c8-4b56-9131-56abaa90694f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:02.537658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-0865af89-89c8-4b56-9131-56abaa90694f None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:02.538854 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0865af89-89c8-4b56-9131-56abaa90694f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:02.546801 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1338/2646] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:32:02.550740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865af89-89c8-4b56-9131-56abaa90694f None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:02.559989 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865af89-89c8-4b56-9131-56abaa90694f None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:02.564573 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69172541-5a76-4df0-92d8-3b46d4f39b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:02.565320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69172541-5a76-4df0-92d8-3b46d4f39b62 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:02.565475 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69172541-5a76-4df0-92d8-3b46d4f39b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:02.565731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69172541-5a76-4df0-92d8-3b46d4f39b62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:02.566342 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69172541-5a76-4df0-92d8-3b46d4f39b62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:32:02.566642 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69172541-5a76-4df0-92d8-3b46d4f39b62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:02.568229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69172541-5a76-4df0-92d8-3b46d4f39b62 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:02.575235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:02.580444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1339/2647] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:32:02.584663 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69172541-5a76-4df0-92d8-3b46d4f39b62 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:02.594661 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69172541-5a76-4df0-92d8-3b46d4f39b62 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:02.599745 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7feea5-55f7-4f91-a0ce-99be7f2682cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:02.600319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7feea5-55f7-4f91-a0ce-99be7f2682cd admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:02.600629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7feea5-55f7-4f91-a0ce-99be7f2682cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:02.600927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7feea5-55f7-4f91-a0ce-99be7f2682cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:02.601526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7feea5-55f7-4f91-a0ce-99be7f2682cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:02.603546 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7feea5-55f7-4f91-a0ce-99be7f2682cd admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:02.617068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:02.639287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2449564b-4470-4f56-b636-3709cc34dfd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a5f9b0a423f4dd488baf792de53e6e4'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:02.639948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2449564b-4470-4f56-b636-3709cc34dfd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a5f9b0a423f4dd488baf792de53e6e4'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:02.719163 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1340/2648] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:32:02 2022] POST /identity/v3/projects => generated 367 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:32:02.723530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7feea5-55f7-4f91-a0ce-99be7f2682cd admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:02.725202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-4a7feea5-55f7-4f91-a0ce-99be7f2682cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:02.740706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2449564b-4470-4f56-b636-3709cc34dfd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a5f9b0a423f4dd488baf792de53e6e4'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:02.753628 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2449564b-4470-4f56-b636-3709cc34dfd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a5f9b0a423f4dd488baf792de53e6e4'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:02.760932 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2449564b-4470-4f56-b636-3709cc34dfd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a5f9b0a423f4dd488baf792de53e6e4'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:02.767188 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1309/2649] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:32:02 2022] DELETE /identity/v3/users/5a5f9b0a423f4dd488baf792de53e6e4 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:32:02.791713 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7feea5-55f7-4f91-a0ce-99be7f2682cd admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:02.851530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20ec54-f7b8-47c9-978b-3dd7687b25ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:02.852153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20ec54-f7b8-47c9-978b-3dd7687b25ae admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:02.852443 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20ec54-f7b8-47c9-978b-3dd7687b25ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:02.852751 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20ec54-f7b8-47c9-978b-3dd7687b25ae admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:02.853388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20ec54-f7b8-47c9-978b-3dd7687b25ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:02.855389 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20ec54-f7b8-47c9-978b-3dd7687b25ae admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:02.950605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1341/2650] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:32:02 2022] POST /identity/v3/users => generated 284 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:32:02.951980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449564b-4470-4f56-b636-3709cc34dfd5 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:02.962986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449564b-4470-4f56-b636-3709cc34dfd5 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:02.968510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc487328-8032-4c57-b034-1fe078bb771b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:02.969133 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc487328-8032-4c57-b034-1fe078bb771b admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:02.969412 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc487328-8032-4c57-b034-1fe078bb771b admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:02.969681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc487328-8032-4c57-b034-1fe078bb771b admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:02.970319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc487328-8032-4c57-b034-1fe078bb771b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:02.971889 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc487328-8032-4c57-b034-1fe078bb771b admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:02.983023 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1310/2651] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:32:02.987365 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20ec54-f7b8-47c9-978b-3dd7687b25ae admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:02.997791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20ec54-f7b8-47c9-978b-3dd7687b25ae admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:03.002613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519af1b-5016-47ae-b350-308eb56867e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:03.003224 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519af1b-5016-47ae-b350-308eb56867e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.003506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519af1b-5016-47ae-b350-308eb56867e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.003783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519af1b-5016-47ae-b350-308eb56867e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.004438 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d519af1b-5016-47ae-b350-308eb56867e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:03.006093 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d519af1b-5016-47ae-b350-308eb56867e2 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:03.023953 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1342/2652] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:32:03.024149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc487328-8032-4c57-b034-1fe078bb771b admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:03.040864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc487328-8032-4c57-b034-1fe078bb771b admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:03.040864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e171d2c-ee4b-4849-bac9-54ef17068f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:03.040864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e171d2c-ee4b-4849-bac9-54ef17068f2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.040864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e171d2c-ee4b-4849-bac9-54ef17068f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.041248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e171d2c-ee4b-4849-bac9-54ef17068f2c admin admin] PATH_INFO: `/v3/projects/1e4df4a729224453b0772c519d84b28f/users/bfa99f33ba6f4566947990961a9b9ed1/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.056170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519af1b-5016-47ae-b350-308eb56867e2 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:03.057260 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e171d2c-ee4b-4849-bac9-54ef17068f2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4df4a729224453b0772c519d84b28f, user_id=bfa99f33ba6f4566947990961a9b9ed1, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:03.058894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-d519af1b-5016-47ae-b350-308eb56867e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:03.059929 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e171d2c-ee4b-4849-bac9-54ef17068f2c admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:03.077506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519af1b-5016-47ae-b350-308eb56867e2 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:03.083720 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae644f-351c-4ebe-a481-8e2c0aac4567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:03.084556 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae644f-351c-4ebe-a481-8e2c0aac4567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.085037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae644f-351c-4ebe-a481-8e2c0aac4567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.085402 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae644f-351c-4ebe-a481-8e2c0aac4567 admin admin] PATH_INFO: `/v3/projects/579dd6bb120849d7af9e66eea809944c` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.087468 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae644f-351c-4ebe-a481-8e2c0aac4567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=579dd6bb120849d7af9e66eea809944c)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:03.089509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae644f-351c-4ebe-a481-8e2c0aac4567 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:03.152170 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1311/2653] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:03 2022] PUT /identity/v3/projects/1e4df4a729224453b0772c519d84b28f/users/bfa99f33ba6f4566947990961a9b9ed1/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:03.162999 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adae322-afdb-4726-b9a8-0b027616f089 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.163288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adae322-afdb-4726-b9a8-0b027616f089 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.163565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adae322-afdb-4726-b9a8-0b027616f089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.191441 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-9adae322-afdb-4726-b9a8-0b027616f089 None None] MFA Rules not processed for user `bfa99f33ba6f4566947990961a9b9ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:03.192667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9adae322-afdb-4726-b9a8-0b027616f089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:03.193687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-fdae644f-351c-4ebe-a481-8e2c0aac4567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '579dd6bb120849d7af9e66eea809944c', 'request_id': 'req-fdae644f-351c-4ebe-a481-8e2c0aac4567'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:03.199160 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-fdae644f-351c-4ebe-a481-8e2c0aac4567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '579dd6bb120849d7af9e66eea809944c', 'request_id': 'req-fdae644f-351c-4ebe-a481-8e2c0aac4567'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:03.207057 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-fdae644f-351c-4ebe-a481-8e2c0aac4567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '579dd6bb120849d7af9e66eea809944c', 'request_id': 'req-fdae644f-351c-4ebe-a481-8e2c0aac4567'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:03.259877 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1312/2654] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:32:03.264550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adae322-afdb-4726-b9a8-0b027616f089 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:03.297718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:03.301722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-fdae644f-351c-4ebe-a481-8e2c0aac4567 admin admin] The token cache is being invalidate because project 579dd6bb120849d7af9e66eea809944c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:32:03.302015 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-fdae644f-351c-4ebe-a481-8e2c0aac4567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:03.314505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-fdae644f-351c-4ebe-a481-8e2c0aac4567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:03.314505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1343/2655] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:32:03 2022] DELETE /identity/v3/projects/579dd6bb120849d7af9e66eea809944c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:03.314894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:03.318127 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eafb0c-a5d5-4c13-8400-3a881a5be52d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.318416 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eafb0c-a5d5-4c13-8400-3a881a5be52d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.318697 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eafb0c-a5d5-4c13-8400-3a881a5be52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.332055 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adae322-afdb-4726-b9a8-0b027616f089 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:03.337202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83f268-67ae-4cdc-ab2c-447cb1015d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:03.337807 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83f268-67ae-4cdc-ab2c-447cb1015d17 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.338087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83f268-67ae-4cdc-ab2c-447cb1015d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.339498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83f268-67ae-4cdc-ab2c-447cb1015d17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.339626 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-31eafb0c-a5d5-4c13-8400-3a881a5be52d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:03.339736 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83f268-67ae-4cdc-ab2c-447cb1015d17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:03.339933 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-31eafb0c-a5d5-4c13-8400-3a881a5be52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:03.342190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83f268-67ae-4cdc-ab2c-447cb1015d17 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:03.349236 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1344/2656] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:32:03.357318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6d540-c43e-45c3-8487-2b5b9465b8b5 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.357682 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6d540-c43e-45c3-8487-2b5b9465b8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.357980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6d540-c43e-45c3-8487-2b5b9465b8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.377544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-f0c6d540-c43e-45c3-8487-2b5b9465b8b5 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:03.378762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f0c6d540-c43e-45c3-8487-2b5b9465b8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:03.386789 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1345/2657] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:32:03.391517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6d540-c43e-45c3-8487-2b5b9465b8b5 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:03.400315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6d540-c43e-45c3-8487-2b5b9465b8b5 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:03.406480 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23efb7-e5e2-4cd0-bab2-3170441dcdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:03.407086 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23efb7-e5e2-4cd0-bab2-3170441dcdb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.407362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23efb7-e5e2-4cd0-bab2-3170441dcdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.407630 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23efb7-e5e2-4cd0-bab2-3170441dcdb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.408268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23efb7-e5e2-4cd0-bab2-3170441dcdb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:32:03.408569 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23efb7-e5e2-4cd0-bab2-3170441dcdb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:03.410243 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23efb7-e5e2-4cd0-bab2-3170441dcdb7 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:03.421459 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1346/2658] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:32:03.425465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23efb7-e5e2-4cd0-bab2-3170441dcdb7 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:03.434508 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23efb7-e5e2-4cd0-bab2-3170441dcdb7 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:03.439269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a87b0-3fa1-4bc8-83b7-4b9d1c4f07dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:03.439830 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a87b0-3fa1-4bc8-83b7-4b9d1c4f07dd admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.440107 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a87b0-3fa1-4bc8-83b7-4b9d1c4f07dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.440378 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a87b0-3fa1-4bc8-83b7-4b9d1c4f07dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.441013 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a87b0-3fa1-4bc8-83b7-4b9d1c4f07dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:03.442855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a87b0-3fa1-4bc8-83b7-4b9d1c4f07dd admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:03.478887 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1313/2659] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:32:03 2022] POST /identity/v3/projects => generated 365 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:32:03.483120 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83f268-67ae-4cdc-ab2c-447cb1015d17 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:03.484911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-bb83f268-67ae-4cdc-ab2c-447cb1015d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:03.550060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83f268-67ae-4cdc-ab2c-447cb1015d17 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:03.554759 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c57abc-6b5c-427c-833a-6b06092f83ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:03.555359 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c57abc-6b5c-427c-833a-6b06092f83ce admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.555641 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c57abc-6b5c-427c-833a-6b06092f83ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.555919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c57abc-6b5c-427c-833a-6b06092f83ce admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.556585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c57abc-6b5c-427c-833a-6b06092f83ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:03.558580 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c57abc-6b5c-427c-833a-6b06092f83ce admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:03.565601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1347/2660] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:32:03 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:32:03.569746 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a87b0-3fa1-4bc8-83b7-4b9d1c4f07dd admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:03.632257 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a87b0-3fa1-4bc8-83b7-4b9d1c4f07dd admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:03.637506 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7b96b-9afa-4fdf-bac3-ab95c4fbd476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:03.638400 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7b96b-9afa-4fdf-bac3-ab95c4fbd476 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.638690 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7b96b-9afa-4fdf-bac3-ab95c4fbd476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.638979 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7b96b-9afa-4fdf-bac3-ab95c4fbd476 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.639603 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7b96b-9afa-4fdf-bac3-ab95c4fbd476 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:03.641559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7b96b-9afa-4fdf-bac3-ab95c4fbd476 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:03.656221 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1314/2661] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:32:03 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:32:03.660297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c57abc-6b5c-427c-833a-6b06092f83ce admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:03.670300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c57abc-6b5c-427c-833a-6b06092f83ce admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:03.675206 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537e193-e1d4-4454-9616-5eb37a2381bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:03.675788 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537e193-e1d4-4454-9616-5eb37a2381bc admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.676076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537e193-e1d4-4454-9616-5eb37a2381bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.676353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537e193-e1d4-4454-9616-5eb37a2381bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.677136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537e193-e1d4-4454-9616-5eb37a2381bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:03.682849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537e193-e1d4-4454-9616-5eb37a2381bc admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:03.689636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1315/2662] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:32:03.693537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537e193-e1d4-4454-9616-5eb37a2381bc admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:03.703730 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537e193-e1d4-4454-9616-5eb37a2381bc admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:03.708310 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa25ae8-3163-486f-addd-793c6e652123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:03.715945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa25ae8-3163-486f-addd-793c6e652123 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.715945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa25ae8-3163-486f-addd-793c6e652123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.715945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa25ae8-3163-486f-addd-793c6e652123 admin admin] PATH_INFO: `/v3/projects/f548b356076c43e7a71df329c586ca8e/users/36ae74977a314572ac90f8bfa434332b/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.725063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa25ae8-3163-486f-addd-793c6e652123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f548b356076c43e7a71df329c586ca8e, user_id=36ae74977a314572ac90f8bfa434332b, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:03.727847 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa25ae8-3163-486f-addd-793c6e652123 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:03.769831 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1348/2663] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:32:03 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:32:03.774011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7b96b-9afa-4fdf-bac3-ab95c4fbd476 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:03.783655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7b96b-9afa-4fdf-bac3-ab95c4fbd476 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:03.789411 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d7d82-25d4-4808-a726-c946091aaa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:03.790005 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d7d82-25d4-4808-a726-c946091aaa05 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.790289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d7d82-25d4-4808-a726-c946091aaa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.790566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d7d82-25d4-4808-a726-c946091aaa05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.791208 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d7d82-25d4-4808-a726-c946091aaa05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:03.792610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1316/2664] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:03 2022] PUT /identity/v3/projects/f548b356076c43e7a71df329c586ca8e/users/36ae74977a314572ac90f8bfa434332b/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:32:03.792838 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d7d82-25d4-4808-a726-c946091aaa05 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:03.801280 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa25ae8-3163-486f-addd-793c6e652123 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:03.803355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1349/2665] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:32:03.807235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d7d82-25d4-4808-a726-c946091aaa05 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:03.817984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d7d82-25d4-4808-a726-c946091aaa05 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:03.866817 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa25ae8-3163-486f-addd-793c6e652123 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:03.871880 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f16e7a-6af6-4ba1-b53e-7a47c5f8bfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:03.872472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f16e7a-6af6-4ba1-b53e-7a47c5f8bfee admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.872787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f16e7a-6af6-4ba1-b53e-7a47c5f8bfee admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.873068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f16e7a-6af6-4ba1-b53e-7a47c5f8bfee admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.873734 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f16e7a-6af6-4ba1-b53e-7a47c5f8bfee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:03.875307 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f16e7a-6af6-4ba1-b53e-7a47c5f8bfee admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:03.875950 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05312edc-516e-4dd2-9598-0f821280495b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:03.876585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05312edc-516e-4dd2-9598-0f821280495b admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.876898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05312edc-516e-4dd2-9598-0f821280495b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.877175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05312edc-516e-4dd2-9598-0f821280495b admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.877809 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05312edc-516e-4dd2-9598-0f821280495b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:03.879428 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05312edc-516e-4dd2-9598-0f821280495b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:03.885255 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1317/2666] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:32:03.889348 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f16e7a-6af6-4ba1-b53e-7a47c5f8bfee admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:03.889554 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1350/2667] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:32:03.893192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05312edc-516e-4dd2-9598-0f821280495b admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:03.899556 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f16e7a-6af6-4ba1-b53e-7a47c5f8bfee admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:03.903576 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05312edc-516e-4dd2-9598-0f821280495b admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:03.905174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9df576-a24d-4595-abc1-6f063103f75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:03.905810 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9df576-a24d-4595-abc1-6f063103f75d admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.906092 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9df576-a24d-4595-abc1-6f063103f75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.906366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9df576-a24d-4595-abc1-6f063103f75d admin admin] PATH_INFO: `/v3/projects/f548b356076c43e7a71df329c586ca8e/users/36ae74977a314572ac90f8bfa434332b/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.908210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca2ece-4735-4190-b907-fd12d3c043c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:03.908362 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9df576-a24d-4595-abc1-6f063103f75d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f548b356076c43e7a71df329c586ca8e, user_id=36ae74977a314572ac90f8bfa434332b, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:03.908870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca2ece-4735-4190-b907-fd12d3c043c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.909154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca2ece-4735-4190-b907-fd12d3c043c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.909432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca2ece-4735-4190-b907-fd12d3c043c2 admin admin] PATH_INFO: `/v3/projects/59704ec2c2fd427b80ebb60a8a9f9b6b/users/15271a6b8ca44359be06059f365d5390/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.910878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9df576-a24d-4595-abc1-6f063103f75d admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:03.921983 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1318/2668] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:03 2022] PUT /identity/v3/projects/f548b356076c43e7a71df329c586ca8e/users/36ae74977a314572ac90f8bfa434332b/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:32:03.924388 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca2ece-4735-4190-b907-fd12d3c043c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59704ec2c2fd427b80ebb60a8a9f9b6b, user_id=15271a6b8ca44359be06059f365d5390, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:03.927045 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca2ece-4735-4190-b907-fd12d3c043c2 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:03.932870 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c786c-e0fa-4157-9c74-5a334e15b0e5 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:03.933225 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c786c-e0fa-4157-9c74-5a334e15b0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:03.933570 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c786c-e0fa-4157-9c74-5a334e15b0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:03.953982 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:03.962454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-d22c786c-e0fa-4157-9c74-5a334e15b0e5 None None] MFA Rules not processed for user `36ae74977a314572ac90f8bfa434332b`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:03.963649 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d22c786c-e0fa-4157-9c74-5a334e15b0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:03.998371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1351/2669] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:03 2022] PUT /identity/v3/projects/59704ec2c2fd427b80ebb60a8a9f9b6b/users/15271a6b8ca44359be06059f365d5390/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:32:04.010158 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655abe48-c565-41bd-ace6-928b721bbfda None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:04.010514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655abe48-c565-41bd-ace6-928b721bbfda None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:04.010870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655abe48-c565-41bd-ace6-928b721bbfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:04.017890 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:04.037951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1319/2670] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:32:04.039064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-655abe48-c565-41bd-ace6-928b721bbfda None None] MFA Rules not processed for user `15271a6b8ca44359be06059f365d5390`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:04.040359 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-655abe48-c565-41bd-ace6-928b721bbfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:04.049890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c786c-e0fa-4157-9c74-5a334e15b0e5 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:04.051259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d22c786c-e0fa-4157-9c74-5a334e15b0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:04.107988 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c786c-e0fa-4157-9c74-5a334e15b0e5 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:04.110322 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1352/2671] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:04.114808 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655abe48-c565-41bd-ace6-928b721bbfda None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:04.161212 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2296e19-8320-46a9-9c97-04a8d782d4bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:04.161820 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2296e19-8320-46a9-9c97-04a8d782d4bf service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:04.162107 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2296e19-8320-46a9-9c97-04a8d782d4bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:04.162375 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2296e19-8320-46a9-9c97-04a8d782d4bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:04.176878 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655abe48-c565-41bd-ace6-928b721bbfda None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:04.182155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32769fe4-5cc2-4b40-bffe-6729bb0784fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:04.186504 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32769fe4-5cc2-4b40-bffe-6729bb0784fe admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:04.186504 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32769fe4-5cc2-4b40-bffe-6729bb0784fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:04.186504 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32769fe4-5cc2-4b40-bffe-6729bb0784fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:04.186504 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32769fe4-5cc2-4b40-bffe-6729bb0784fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:04.186504 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32769fe4-5cc2-4b40-bffe-6729bb0784fe admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:04.224704 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2296e19-8320-46a9-9c97-04a8d782d4bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:04.226127 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2296e19-8320-46a9-9c97-04a8d782d4bf service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:04.231050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=100 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:04.235678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1320/2672] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:32:04.302889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1353/2673] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:32:04 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:32:04.306911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2296e19-8320-46a9-9c97-04a8d782d4bf service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:04.317365 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2296e19-8320-46a9-9c97-04a8d782d4bf service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:04.376969 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a50fd-f67d-4d35-8819-926f871540d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:04.377493 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a50fd-f67d-4d35-8819-926f871540d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:04.377773 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a50fd-f67d-4d35-8819-926f871540d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:04.378043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a50fd-f67d-4d35-8819-926f871540d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:04.378666 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a50fd-f67d-4d35-8819-926f871540d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:04.380571 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a50fd-f67d-4d35-8819-926f871540d5 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:04.463744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1321/2674] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:32:04 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:32:04.467549 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32769fe4-5cc2-4b40-bffe-6729bb0784fe admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:04.478565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32769fe4-5cc2-4b40-bffe-6729bb0784fe admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:04.484400 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f283e41-55e4-4fe4-a9a0-17c5b90a120f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:04.485050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f283e41-55e4-4fe4-a9a0-17c5b90a120f admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:04.485453 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f283e41-55e4-4fe4-a9a0-17c5b90a120f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:04.485735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f283e41-55e4-4fe4-a9a0-17c5b90a120f admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:04.486380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f283e41-55e4-4fe4-a9a0-17c5b90a120f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:04.488060 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f283e41-55e4-4fe4-a9a0-17c5b90a120f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:04.499893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1354/2675] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:32:04.504248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a50fd-f67d-4d35-8819-926f871540d5 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:04.515243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a50fd-f67d-4d35-8819-926f871540d5 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:04.520294 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b2575-ae88-49a1-a007-01d85c218319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:04.520972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b2575-ae88-49a1-a007-01d85c218319 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:04.521256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b2575-ae88-49a1-a007-01d85c218319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:04.521528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b2575-ae88-49a1-a007-01d85c218319 admin admin] PATH_INFO: `/v3/projects/a94eb6e0b204429e83ad3480585d9eac/users/b71e6e57f8f34e9598e0a8c2e2e7a228/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:04.537305 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b2575-ae88-49a1-a007-01d85c218319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a94eb6e0b204429e83ad3480585d9eac, user_id=b71e6e57f8f34e9598e0a8c2e2e7a228, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:04.539906 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b2575-ae88-49a1-a007-01d85c218319 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:04.601886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f283e41-55e4-4fe4-a9a0-17c5b90a120f admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:04.656505 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1322/2676] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:04 2022] PUT /identity/v3/projects/a94eb6e0b204429e83ad3480585d9eac/users/b71e6e57f8f34e9598e0a8c2e2e7a228/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:04.660579 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b2575-ae88-49a1-a007-01d85c218319 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:04.669678 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f283e41-55e4-4fe4-a9a0-17c5b90a120f admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:04.689824 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:04.719977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456f43c-8540-48bb-b91a-5205c1ab9a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:04.720704 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456f43c-8540-48bb-b91a-5205c1ab9a7b service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:04.721068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456f43c-8540-48bb-b91a-5205c1ab9a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:04.721417 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456f43c-8540-48bb-b91a-5205c1ab9a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:04.723789 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9456f43c-8540-48bb-b91a-5205c1ab9a7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:04.726184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b2575-ae88-49a1-a007-01d85c218319 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:04.731049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bef43-0c0e-4edd-b85c-f8a5144172ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:04.733965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bef43-0c0e-4edd-b85c-f8a5144172ce admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:04.733965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bef43-0c0e-4edd-b85c-f8a5144172ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:04.733965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bef43-0c0e-4edd-b85c-f8a5144172ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:04.733965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bef43-0c0e-4edd-b85c-f8a5144172ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:04.735221 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bef43-0c0e-4edd-b85c-f8a5144172ce admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:04.745121 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1323/2677] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:32:04.749389 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bef43-0c0e-4edd-b85c-f8a5144172ce admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:04.759341 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bef43-0c0e-4edd-b85c-f8a5144172ce admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:04.764541 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc9dfe-33f7-4a3b-a221-b939c4465efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:04.765258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc9dfe-33f7-4a3b-a221-b939c4465efb admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:04.765543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc9dfe-33f7-4a3b-a221-b939c4465efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:04.765818 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc9dfe-33f7-4a3b-a221-b939c4465efb admin admin] PATH_INFO: `/v3/projects/a94eb6e0b204429e83ad3480585d9eac/users/b71e6e57f8f34e9598e0a8c2e2e7a228/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:04.767746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc9dfe-33f7-4a3b-a221-b939c4465efb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a94eb6e0b204429e83ad3480585d9eac, user_id=b71e6e57f8f34e9598e0a8c2e2e7a228, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:04.770398 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc9dfe-33f7-4a3b-a221-b939c4465efb admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:04.781537 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1324/2678] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:04 2022] PUT /identity/v3/projects/a94eb6e0b204429e83ad3480585d9eac/users/b71e6e57f8f34e9598e0a8c2e2e7a228/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:32:04.786706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9456f43c-8540-48bb-b91a-5205c1ab9a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:04.788150 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9456f43c-8540-48bb-b91a-5205c1ab9a7b service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:04.791634 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9939c1-e875-4350-9b0f-5f23580c6b22 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:04.791921 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9939c1-e875-4350-9b0f-5f23580c6b22 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:04.792196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9939c1-e875-4350-9b0f-5f23580c6b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:04.801157 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1355/2679] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:32:04.824982 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-4d9939c1-e875-4350-9b0f-5f23580c6b22 None None] MFA Rules not processed for user `b71e6e57f8f34e9598e0a8c2e2e7a228`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:04.826304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4d9939c1-e875-4350-9b0f-5f23580c6b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:04.867043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:04.901805 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1325/2680] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:32:04.954552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456f43c-8540-48bb-b91a-5205c1ab9a7b service cinder] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:04.956040 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9456f43c-8540-48bb-b91a-5205c1ab9a7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:04.968504 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456f43c-8540-48bb-b91a-5205c1ab9a7b service cinder] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:05.018151 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac6627-e1b0-45e1-b693-fc40da79a344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:05.018748 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac6627-e1b0-45e1-b693-fc40da79a344 service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:05.019032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac6627-e1b0-45e1-b693-fc40da79a344 service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:05.019304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac6627-e1b0-45e1-b693-fc40da79a344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:05.028700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac6627-e1b0-45e1-b693-fc40da79a344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:05.030090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac6627-e1b0-45e1-b693-fc40da79a344 service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:05.039291 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1356/2681] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:32:05.371830 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9939c1-e875-4350-9b0f-5f23580c6b22 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:05.381839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9939c1-e875-4350-9b0f-5f23580c6b22 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:05.385472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ad7e9-6059-4116-af72-a3135a95c8ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:05.389671 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ad7e9-6059-4116-af72-a3135a95c8ee service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:05.389671 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ad7e9-6059-4116-af72-a3135a95c8ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:05.389671 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ad7e9-6059-4116-af72-a3135a95c8ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:05.444357 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ad7e9-6059-4116-af72-a3135a95c8ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:05.445941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ad7e9-6059-4116-af72-a3135a95c8ee service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:05.456159 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1326/2682] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:32:07.584577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac6627-e1b0-45e1-b693-fc40da79a344 service cinder] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:07.586300 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-00ac6627-e1b0-45e1-b693-fc40da79a344 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:07.655091 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac6627-e1b0-45e1-b693-fc40da79a344 service cinder] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:07.660001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a3a95-4f06-4125-8bc1-e6689446387c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:07.660629 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a3a95-4f06-4125-8bc1-e6689446387c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:07.660950 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a3a95-4f06-4125-8bc1-e6689446387c admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:07.661219 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a3a95-4f06-4125-8bc1-e6689446387c admin admin] PATH_INFO: `/v3/users/17cdefa08847443697e485f11c3c7df0` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:07.662346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a3a95-4f06-4125-8bc1-e6689446387c admin admin] RBAC: Authorizing `identity:delete_user(user_id=17cdefa08847443697e485f11c3c7df0)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:07.664326 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a3a95-4f06-4125-8bc1-e6689446387c admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:07.691577 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:07.950522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f74a3a95-4f06-4125-8bc1-e6689446387c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17cdefa08847443697e485f11c3c7df0'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:08.054517 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f74a3a95-4f06-4125-8bc1-e6689446387c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17cdefa08847443697e485f11c3c7df0'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:08.055209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f74a3a95-4f06-4125-8bc1-e6689446387c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17cdefa08847443697e485f11c3c7df0'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:08.064089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f74a3a95-4f06-4125-8bc1-e6689446387c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17cdefa08847443697e485f11c3c7df0'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:08.078266 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f74a3a95-4f06-4125-8bc1-e6689446387c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17cdefa08847443697e485f11c3c7df0'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:08.084092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1357/2683] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:32:07 2022] DELETE /identity/v3/users/17cdefa08847443697e485f11c3c7df0 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:08.089048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f93022-b9cf-43bf-918c-8c845ce39fcd None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:08.089520 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f93022-b9cf-43bf-918c-8c845ce39fcd None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:08.089623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f93022-b9cf-43bf-918c-8c845ce39fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:08.109272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-36f93022-b9cf-43bf-918c-8c845ce39fcd None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:08.110463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-36f93022-b9cf-43bf-918c-8c845ce39fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:08.168831 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1327/2684] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:08.179133 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a3a95-4f06-4125-8bc1-e6689446387c admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:08.180552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-f74a3a95-4f06-4125-8bc1-e6689446387c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:08.243073 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a3a95-4f06-4125-8bc1-e6689446387c admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:08.246969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b03fb-2989-4da6-9026-09c0bf6ee522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:08.247569 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b03fb-2989-4da6-9026-09c0bf6ee522 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:08.247887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b03fb-2989-4da6-9026-09c0bf6ee522 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:08.248134 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b03fb-2989-4da6-9026-09c0bf6ee522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:08.257559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b03fb-2989-4da6-9026-09c0bf6ee522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:08.258984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b03fb-2989-4da6-9026-09c0bf6ee522 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:08.271687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1358/2685] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:32:08.679063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f93022-b9cf-43bf-918c-8c845ce39fcd None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:08.680502 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-36f93022-b9cf-43bf-918c-8c845ce39fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:08.692778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f93022-b9cf-43bf-918c-8c845ce39fcd None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:08.697422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:08.698073 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:08.698352 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:08.698623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc admin admin] PATH_INFO: `/v3/projects/22173887a0b44ba79d83bbe8d786e411` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:08.699919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=22173887a0b44ba79d83bbe8d786e411)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:08.702373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:08.835062 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22173887a0b44ba79d83bbe8d786e411', 'request_id': 'req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:08.841046 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22173887a0b44ba79d83bbe8d786e411', 'request_id': 'req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:08.856588 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22173887a0b44ba79d83bbe8d786e411', 'request_id': 'req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:08.856588 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:08.989324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc admin admin] The token cache is being invalidate because project 22173887a0b44ba79d83bbe8d786e411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:32:08.989613 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:08.990251 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:08.991741 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1328/2686] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:32:08 2022] DELETE /identity/v3/projects/22173887a0b44ba79d83bbe8d786e411 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:08.996097 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b03fb-2989-4da6-9026-09c0bf6ee522 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:09.006538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b03fb-2989-4da6-9026-09c0bf6ee522 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:09.062085 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02f491-3a95-4ff5-8b04-3d0021c84ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:09.062705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02f491-3a95-4ff5-8b04-3d0021c84ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:09.062985 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02f491-3a95-4ff5-8b04-3d0021c84ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:09.063260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02f491-3a95-4ff5-8b04-3d0021c84ced admin admin] PATH_INFO: `/v3/users/dffda2559a4343079e1e1e8c77f87f05` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:09.064371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02f491-3a95-4ff5-8b04-3d0021c84ced admin admin] RBAC: Authorizing `identity:delete_user(user_id=dffda2559a4343079e1e1e8c77f87f05)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:09.066332 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02f491-3a95-4ff5-8b04-3d0021c84ced admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:09.337039 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:09.339718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-da02f491-3a95-4ff5-8b04-3d0021c84ced admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dffda2559a4343079e1e1e8c77f87f05'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:09.449600 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-da02f491-3a95-4ff5-8b04-3d0021c84ced admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dffda2559a4343079e1e1e8c77f87f05'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:09.450259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-da02f491-3a95-4ff5-8b04-3d0021c84ced admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dffda2559a4343079e1e1e8c77f87f05'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:09.457948 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-da02f491-3a95-4ff5-8b04-3d0021c84ced admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dffda2559a4343079e1e1e8c77f87f05'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:09.470516 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-da02f491-3a95-4ff5-8b04-3d0021c84ced admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dffda2559a4343079e1e1e8c77f87f05'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:09.476573 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1359/2687] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:32:08 2022] DELETE /identity/v3/users/dffda2559a4343079e1e1e8c77f87f05 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:09.912135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:09.912135 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:09.924827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af3f33-d77c-40d6-b4a8-e0dddebef8bc admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:09.986970 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b23cc1-ef43-4e9f-af67-ce8d53705b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:09.987155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b23cc1-ef43-4e9f-af67-ce8d53705b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:09.987381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b23cc1-ef43-4e9f-af67-ce8d53705b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:09.987653 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b23cc1-ef43-4e9f-af67-ce8d53705b7b admin admin] PATH_INFO: `/v3/projects/0d6bf03ea2ff471f8ae61b1bd3a3eb4c` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:09.988815 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b23cc1-ef43-4e9f-af67-ce8d53705b7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d6bf03ea2ff471f8ae61b1bd3a3eb4c)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:09.990700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b23cc1-ef43-4e9f-af67-ce8d53705b7b admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:10.084031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d9b23cc1-ef43-4e9f-af67-ce8d53705b7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d6bf03ea2ff471f8ae61b1bd3a3eb4c', 'request_id': 'req-d9b23cc1-ef43-4e9f-af67-ce8d53705b7b'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:10.089672 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d9b23cc1-ef43-4e9f-af67-ce8d53705b7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d6bf03ea2ff471f8ae61b1bd3a3eb4c', 'request_id': 'req-d9b23cc1-ef43-4e9f-af67-ce8d53705b7b'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:10.096424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d9b23cc1-ef43-4e9f-af67-ce8d53705b7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d6bf03ea2ff471f8ae61b1bd3a3eb4c', 'request_id': 'req-d9b23cc1-ef43-4e9f-af67-ce8d53705b7b'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:10.180591 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d9b23cc1-ef43-4e9f-af67-ce8d53705b7b admin admin] The token cache is being invalidate because project 0d6bf03ea2ff471f8ae61b1bd3a3eb4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:32:10.181081 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d9b23cc1-ef43-4e9f-af67-ce8d53705b7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:10.181643 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d9b23cc1-ef43-4e9f-af67-ce8d53705b7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:10.183258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1329/2688] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:32:09 2022] DELETE /identity/v3/projects/0d6bf03ea2ff471f8ae61b1bd3a3eb4c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:10.189581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ac0e0-4c3d-4655-aeec-5453c520fac7 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:10.190171 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ac0e0-4c3d-4655-aeec-5453c520fac7 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:10.190715 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ac0e0-4c3d-4655-aeec-5453c520fac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:10.215485 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-d95ac0e0-4c3d-4655-aeec-5453c520fac7 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:10.217494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-d95ac0e0-4c3d-4655-aeec-5453c520fac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:10.281660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1360/2689] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:32:10.290777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d4c49-cb5e-4965-a95d-a03bd9333a5e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:10.291088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d4c49-cb5e-4965-a95d-a03bd9333a5e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:10.291363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d4c49-cb5e-4965-a95d-a03bd9333a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:10.311144 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-dd4d4c49-cb5e-4965-a95d-a03bd9333a5e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:10.312366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-dd4d4c49-cb5e-4965-a95d-a03bd9333a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:10.320209 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1330/2690] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:10.324666 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ac0e0-4c3d-4655-aeec-5453c520fac7 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:10.335233 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ac0e0-4c3d-4655-aeec-5453c520fac7 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:10.343420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03880185-c791-459f-8a15-26bc379d3b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:10.343420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03880185-c791-459f-8a15-26bc379d3b89 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:10.343420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03880185-c791-459f-8a15-26bc379d3b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:10.343420 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03880185-c791-459f-8a15-26bc379d3b89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:10.343802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03880185-c791-459f-8a15-26bc379d3b89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:32:10.343802 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03880185-c791-459f-8a15-26bc379d3b89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:10.345200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03880185-c791-459f-8a15-26bc379d3b89 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:10.357483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1361/2691] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:32:10.362282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d4c49-cb5e-4965-a95d-a03bd9333a5e None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:10.371616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d4c49-cb5e-4965-a95d-a03bd9333a5e None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:10.380212 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0c72f-e599-48be-92d5-5cc5781fbad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:10.380849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0c72f-e599-48be-92d5-5cc5781fbad5 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:10.381167 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0c72f-e599-48be-92d5-5cc5781fbad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:10.381437 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0c72f-e599-48be-92d5-5cc5781fbad5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:10.382108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0c72f-e599-48be-92d5-5cc5781fbad5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:10.383993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0c72f-e599-48be-92d5-5cc5781fbad5 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:10.472328 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1331/2692] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:32:10 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:32:10.477844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03880185-c791-459f-8a15-26bc379d3b89 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:10.484347 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03880185-c791-459f-8a15-26bc379d3b89 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:10.542047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58578d6-ea8b-45ee-a3d1-3b2276982320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:10.542658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58578d6-ea8b-45ee-a3d1-3b2276982320 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:10.542944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58578d6-ea8b-45ee-a3d1-3b2276982320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:10.543224 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58578d6-ea8b-45ee-a3d1-3b2276982320 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:10.543847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58578d6-ea8b-45ee-a3d1-3b2276982320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:10.546043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58578d6-ea8b-45ee-a3d1-3b2276982320 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:10.656674 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1362/2693] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:32:10 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:32:10.660928 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0c72f-e599-48be-92d5-5cc5781fbad5 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:10.671358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0c72f-e599-48be-92d5-5cc5781fbad5 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:10.675995 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84d33d-6a14-47f1-8f5b-a17485dfe02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:10.676588 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84d33d-6a14-47f1-8f5b-a17485dfe02e admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:10.676901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84d33d-6a14-47f1-8f5b-a17485dfe02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:10.677175 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84d33d-6a14-47f1-8f5b-a17485dfe02e admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:10.677822 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84d33d-6a14-47f1-8f5b-a17485dfe02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:10.679406 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84d33d-6a14-47f1-8f5b-a17485dfe02e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:10.689578 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1332/2694] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:32:10.693900 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58578d6-ea8b-45ee-a3d1-3b2276982320 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:10.707375 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58578d6-ea8b-45ee-a3d1-3b2276982320 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:10.709079 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2ad23-fd72-43bb-800f-0a9365e7ad88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:10.709664 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2ad23-fd72-43bb-800f-0a9365e7ad88 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:10.709941 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2ad23-fd72-43bb-800f-0a9365e7ad88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:10.710261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2ad23-fd72-43bb-800f-0a9365e7ad88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:10.711187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2ad23-fd72-43bb-800f-0a9365e7ad88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:10.712791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2ad23-fd72-43bb-800f-0a9365e7ad88 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:10.724180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1363/2695] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:32:10.728406 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84d33d-6a14-47f1-8f5b-a17485dfe02e admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:10.738925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84d33d-6a14-47f1-8f5b-a17485dfe02e admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:10.743470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e541bf-ef0e-490d-ac78-284b7eff7937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:10.744108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e541bf-ef0e-490d-ac78-284b7eff7937 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:10.744389 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e541bf-ef0e-490d-ac78-284b7eff7937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:10.744685 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e541bf-ef0e-490d-ac78-284b7eff7937 admin admin] PATH_INFO: `/v3/projects/fa463c33ef374dbf8f06eddba7a59390/users/7edac7c8aee74284b1f7bdbf055fd1f7/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:10.758813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e541bf-ef0e-490d-ac78-284b7eff7937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa463c33ef374dbf8f06eddba7a59390, user_id=7edac7c8aee74284b1f7bdbf055fd1f7, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:10.761459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e541bf-ef0e-490d-ac78-284b7eff7937 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:10.878624 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1333/2696] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:10 2022] PUT /identity/v3/projects/fa463c33ef374dbf8f06eddba7a59390/users/7edac7c8aee74284b1f7bdbf055fd1f7/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:32:10.889752 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4135ef1-94f7-483a-9e22-5c25efd31e6c None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:10.890041 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4135ef1-94f7-483a-9e22-5c25efd31e6c None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:10.890317 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4135ef1-94f7-483a-9e22-5c25efd31e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:10.917143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-d4135ef1-94f7-483a-9e22-5c25efd31e6c None None] MFA Rules not processed for user `7edac7c8aee74284b1f7bdbf055fd1f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:10.918342 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-d4135ef1-94f7-483a-9e22-5c25efd31e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:10.930699 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:10.986752 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1364/2697] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:10.997105 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e541bf-ef0e-490d-ac78-284b7eff7937 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:11.017694 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:11.051965 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e541bf-ef0e-490d-ac78-284b7eff7937 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:11.056665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5dcba-c6a9-4927-8925-35ea01d42cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:11.057296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5dcba-c6a9-4927-8925-35ea01d42cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:11.057561 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5dcba-c6a9-4927-8925-35ea01d42cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:11.057833 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5dcba-c6a9-4927-8925-35ea01d42cf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:11.058446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5dcba-c6a9-4927-8925-35ea01d42cf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:11.060856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5dcba-c6a9-4927-8925-35ea01d42cf2 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:11.131076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1334/2698] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:32:10 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:32:11.135120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4135ef1-94f7-483a-9e22-5c25efd31e6c None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:11.147235 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4135ef1-94f7-483a-9e22-5c25efd31e6c None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:11.205606 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02df991-0d99-4d7e-bcb9-0735f1872118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:11.206284 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02df991-0d99-4d7e-bcb9-0735f1872118 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:11.206483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02df991-0d99-4d7e-bcb9-0735f1872118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:11.206760 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02df991-0d99-4d7e-bcb9-0735f1872118 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:11.207377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02df991-0d99-4d7e-bcb9-0735f1872118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:11.209320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02df991-0d99-4d7e-bcb9-0735f1872118 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:11.283437 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1365/2699] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:32:11 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:32:11.287518 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5dcba-c6a9-4927-8925-35ea01d42cf2 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:11.298543 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5dcba-c6a9-4927-8925-35ea01d42cf2 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:11.303132 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191db6c-8103-4442-99e2-339a7599f5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:11.303718 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191db6c-8103-4442-99e2-339a7599f5c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:11.303996 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191db6c-8103-4442-99e2-339a7599f5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:11.304266 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191db6c-8103-4442-99e2-339a7599f5c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:11.304955 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191db6c-8103-4442-99e2-339a7599f5c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:11.306532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191db6c-8103-4442-99e2-339a7599f5c2 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:11.317818 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1335/2700] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:32:11.322025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02df991-0d99-4d7e-bcb9-0735f1872118 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:11.332852 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02df991-0d99-4d7e-bcb9-0735f1872118 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:11.337655 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcbb52-b793-4e00-92dd-e14a47a00592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:11.338289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcbb52-b793-4e00-92dd-e14a47a00592 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:11.338567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcbb52-b793-4e00-92dd-e14a47a00592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:11.338841 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcbb52-b793-4e00-92dd-e14a47a00592 admin admin] PATH_INFO: `/v3/projects/eddc0596aa594795ab42424ada9a2267/users/dfe3a8aafc3c4d1ebefe98cd67fb2072/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:11.352992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcbb52-b793-4e00-92dd-e14a47a00592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eddc0596aa594795ab42424ada9a2267, user_id=dfe3a8aafc3c4d1ebefe98cd67fb2072, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:11.355590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcbb52-b793-4e00-92dd-e14a47a00592 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:11.421178 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1366/2701] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:11 2022] PUT /identity/v3/projects/eddc0596aa594795ab42424ada9a2267/users/dfe3a8aafc3c4d1ebefe98cd67fb2072/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:11.425187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191db6c-8103-4442-99e2-339a7599f5c2 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:11.435190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191db6c-8103-4442-99e2-339a7599f5c2 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:11.490158 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a186df7-e084-4515-ad5f-896dfd0943f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:11.490753 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a186df7-e084-4515-ad5f-896dfd0943f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:11.491031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a186df7-e084-4515-ad5f-896dfd0943f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:11.491300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a186df7-e084-4515-ad5f-896dfd0943f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:11.491941 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a186df7-e084-4515-ad5f-896dfd0943f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:11.493530 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a186df7-e084-4515-ad5f-896dfd0943f2 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:11.503656 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1336/2702] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:32:11.507792 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcbb52-b793-4e00-92dd-e14a47a00592 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:11.518123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcbb52-b793-4e00-92dd-e14a47a00592 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:11.522742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47993d6d-fd06-48cc-808e-72717d2ce384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:11.523379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47993d6d-fd06-48cc-808e-72717d2ce384 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:11.523658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47993d6d-fd06-48cc-808e-72717d2ce384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:11.523930 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47993d6d-fd06-48cc-808e-72717d2ce384 admin admin] PATH_INFO: `/v3/projects/eddc0596aa594795ab42424ada9a2267/users/dfe3a8aafc3c4d1ebefe98cd67fb2072/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:11.525857 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47993d6d-fd06-48cc-808e-72717d2ce384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eddc0596aa594795ab42424ada9a2267, user_id=dfe3a8aafc3c4d1ebefe98cd67fb2072, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:11.528503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47993d6d-fd06-48cc-808e-72717d2ce384 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:11.538778 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1367/2703] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:11 2022] PUT /identity/v3/projects/eddc0596aa594795ab42424ada9a2267/users/dfe3a8aafc3c4d1ebefe98cd67fb2072/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:32:11.549470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd162d-9cc9-4400-b9f2-9713f511d10c None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:11.549764 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd162d-9cc9-4400-b9f2-9713f511d10c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:11.550041 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bd162d-9cc9-4400-b9f2-9713f511d10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:11.579094 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-26bd162d-9cc9-4400-b9f2-9713f511d10c None None] MFA Rules not processed for user `dfe3a8aafc3c4d1ebefe98cd67fb2072`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:11.579094 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-26bd162d-9cc9-4400-b9f2-9713f511d10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:11.646793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1337/2704] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:11.658638 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47993d6d-fd06-48cc-808e-72717d2ce384 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:11.660422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-47993d6d-fd06-48cc-808e-72717d2ce384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:11.719588 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47993d6d-fd06-48cc-808e-72717d2ce384 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:11.723462 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647aab4-ef7f-4e0a-98f8-cbef9083e2fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:11.724083 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647aab4-ef7f-4e0a-98f8-cbef9083e2fc service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:11.724384 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647aab4-ef7f-4e0a-98f8-cbef9083e2fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:11.724707 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647aab4-ef7f-4e0a-98f8-cbef9083e2fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:11.734896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647aab4-ef7f-4e0a-98f8-cbef9083e2fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:11.736371 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647aab4-ef7f-4e0a-98f8-cbef9083e2fc service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:11.746251 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1368/2705] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:32:11.942074 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd162d-9cc9-4400-b9f2-9713f511d10c None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:11.952201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bd162d-9cc9-4400-b9f2-9713f511d10c None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:11.955795 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df01c26-9058-434b-b326-73042bdd590d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:11.956381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df01c26-9058-434b-b326-73042bdd590d service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:11.956690 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df01c26-9058-434b-b326-73042bdd590d service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:11.956964 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df01c26-9058-434b-b326-73042bdd590d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:11.971103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:12.015298 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df01c26-9058-434b-b326-73042bdd590d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:12.017295 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df01c26-9058-434b-b326-73042bdd590d service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:12.025919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1338/2706] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:32:14.761425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647aab4-ef7f-4e0a-98f8-cbef9083e2fc service cinder] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:14.762859 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-6647aab4-ef7f-4e0a-98f8-cbef9083e2fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:14.776886 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:14.834559 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647aab4-ef7f-4e0a-98f8-cbef9083e2fc service cinder] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:14.839362 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d138f43-2142-42e8-a35d-b61e45a6aadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:14.839987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d138f43-2142-42e8-a35d-b61e45a6aadb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:14.840268 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d138f43-2142-42e8-a35d-b61e45a6aadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:14.840541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d138f43-2142-42e8-a35d-b61e45a6aadb admin admin] PATH_INFO: `/v3/users/bfa99f33ba6f4566947990961a9b9ed1` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:14.841793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d138f43-2142-42e8-a35d-b61e45a6aadb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfa99f33ba6f4566947990961a9b9ed1)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:14.844015 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d138f43-2142-42e8-a35d-b61e45a6aadb admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:15.039831 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-8d138f43-2142-42e8-a35d-b61e45a6aadb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfa99f33ba6f4566947990961a9b9ed1'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:15.123144 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-8d138f43-2142-42e8-a35d-b61e45a6aadb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfa99f33ba6f4566947990961a9b9ed1'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:15.123765 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-8d138f43-2142-42e8-a35d-b61e45a6aadb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfa99f33ba6f4566947990961a9b9ed1'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:15.132229 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-8d138f43-2142-42e8-a35d-b61e45a6aadb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfa99f33ba6f4566947990961a9b9ed1'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:15.144021 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-8d138f43-2142-42e8-a35d-b61e45a6aadb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfa99f33ba6f4566947990961a9b9ed1'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:15.149792 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1369/2707] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:32:14 2022] DELETE /identity/v3/users/bfa99f33ba6f4566947990961a9b9ed1 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:15.154325 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf7084-4fed-44e8-bc45-2206757cb47c None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:15.154795 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf7084-4fed-44e8-bc45-2206757cb47c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:15.154897 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf7084-4fed-44e8-bc45-2206757cb47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:15.175320 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-40cf7084-4fed-44e8-bc45-2206757cb47c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:15.176518 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-40cf7084-4fed-44e8-bc45-2206757cb47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:15.235370 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1339/2708] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:32:15.245824 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d138f43-2142-42e8-a35d-b61e45a6aadb admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:15.247285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-8d138f43-2142-42e8-a35d-b61e45a6aadb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:15.304356 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d138f43-2142-42e8-a35d-b61e45a6aadb admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:15.307976 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce56e7d-a4d6-446c-8735-878cfb249d25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:15.308574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce56e7d-a4d6-446c-8735-878cfb249d25 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:15.308885 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce56e7d-a4d6-446c-8735-878cfb249d25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:15.309161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce56e7d-a4d6-446c-8735-878cfb249d25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:15.318391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce56e7d-a4d6-446c-8735-878cfb249d25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:15.319804 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce56e7d-a4d6-446c-8735-878cfb249d25 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:15.329123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1370/2709] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:32:15.699566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf7084-4fed-44e8-bc45-2206757cb47c None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:15.701123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-40cf7084-4fed-44e8-bc45-2206757cb47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:15.714677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf7084-4fed-44e8-bc45-2206757cb47c None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:15.719419 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d8ef1-0bdc-4daa-9181-b5a783860a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:15.720055 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d8ef1-0bdc-4daa-9181-b5a783860a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:15.720338 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d8ef1-0bdc-4daa-9181-b5a783860a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:15.720618 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d8ef1-0bdc-4daa-9181-b5a783860a4e admin admin] PATH_INFO: `/v3/users/7edac7c8aee74284b1f7bdbf055fd1f7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:15.722526 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d8ef1-0bdc-4daa-9181-b5a783860a4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7edac7c8aee74284b1f7bdbf055fd1f7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:15.724462 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d8ef1-0bdc-4daa-9181-b5a783860a4e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:15.891395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a91d8ef1-0bdc-4daa-9181-b5a783860a4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7edac7c8aee74284b1f7bdbf055fd1f7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:15.892165 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a91d8ef1-0bdc-4daa-9181-b5a783860a4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7edac7c8aee74284b1f7bdbf055fd1f7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:15.934598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a91d8ef1-0bdc-4daa-9181-b5a783860a4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7edac7c8aee74284b1f7bdbf055fd1f7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:15.946926 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a91d8ef1-0bdc-4daa-9181-b5a783860a4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7edac7c8aee74284b1f7bdbf055fd1f7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:15.953744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a91d8ef1-0bdc-4daa-9181-b5a783860a4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7edac7c8aee74284b1f7bdbf055fd1f7'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:15.960140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1340/2710] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:32:15 2022] DELETE /identity/v3/users/7edac7c8aee74284b1f7bdbf055fd1f7 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:15.964965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506cdd8-3930-4b5a-aee4-d92a43136e76 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:15.965446 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506cdd8-3930-4b5a-aee4-d92a43136e76 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:15.965569 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506cdd8-3930-4b5a-aee4-d92a43136e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:15.984846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-5506cdd8-3930-4b5a-aee4-d92a43136e76 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:15.986058 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-5506cdd8-3930-4b5a-aee4-d92a43136e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:16.006469 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d8ef1-0bdc-4daa-9181-b5a783860a4e admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:16.008044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a91d8ef1-0bdc-4daa-9181-b5a783860a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:16.034092 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:16.050780 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:16.075783 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1371/2711] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:16.086418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506cdd8-3930-4b5a-aee4-d92a43136e76 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:16.093450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-5506cdd8-3930-4b5a-aee4-d92a43136e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:16.098335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d8ef1-0bdc-4daa-9181-b5a783860a4e admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:16.103562 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:16.104265 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:16.104617 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:16.105037 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe admin admin] PATH_INFO: `/v3/projects/1e4df4a729224453b0772c519d84b28f` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:16.106301 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e4df4a729224453b0772c519d84b28f)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:16.108250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:16.153615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506cdd8-3930-4b5a-aee4-d92a43136e76 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:16.157267 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cb095-b439-48ce-99a0-33f16606e3a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:16.180570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cb095-b439-48ce-99a0-33f16606e3a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:16.180570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cb095-b439-48ce-99a0-33f16606e3a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:16.180570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cb095-b439-48ce-99a0-33f16606e3a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:16.180570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7cb095-b439-48ce-99a0-33f16606e3a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:16.180570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7cb095-b439-48ce-99a0-33f16606e3a1 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:16.180570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1372/2712] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:32:16.264044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e4df4a729224453b0772c519d84b28f', 'request_id': 'req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:16.269709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e4df4a729224453b0772c519d84b28f', 'request_id': 'req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:16.275981 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e4df4a729224453b0772c519d84b28f', 'request_id': 'req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:16.392028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe admin admin] The token cache is being invalidate because project 1e4df4a729224453b0772c519d84b28f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:32:16.393120 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:16.394112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:16.396096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1341/2713] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:32:16 2022] DELETE /identity/v3/projects/1e4df4a729224453b0772c519d84b28f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:32:16.400120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cb095-b439-48ce-99a0-33f16606e3a1 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:16.411233 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cb095-b439-48ce-99a0-33f16606e3a1 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:16.470309 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb232c1-441a-4565-b5f4-ca3756aa065e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:16.471686 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb232c1-441a-4565-b5f4-ca3756aa065e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:16.472076 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb232c1-441a-4565-b5f4-ca3756aa065e admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:16.472344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb232c1-441a-4565-b5f4-ca3756aa065e admin admin] PATH_INFO: `/v3/users/36ae74977a314572ac90f8bfa434332b` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:16.473574 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb232c1-441a-4565-b5f4-ca3756aa065e admin admin] RBAC: Authorizing `identity:delete_user(user_id=36ae74977a314572ac90f8bfa434332b)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:16.475780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb232c1-441a-4565-b5f4-ca3756aa065e admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:16.656588 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:16.658055 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:16.672353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d88f60-b287-4e44-9c6f-9ed25e3ac4fe admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:16.678597 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b4ccc-15a5-430e-85ef-018139b48f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:16.679794 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b4ccc-15a5-430e-85ef-018139b48f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:16.679794 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b4ccc-15a5-430e-85ef-018139b48f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:16.680070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b4ccc-15a5-430e-85ef-018139b48f88 admin admin] PATH_INFO: `/v3/projects/fa463c33ef374dbf8f06eddba7a59390` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:16.681394 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b4ccc-15a5-430e-85ef-018139b48f88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa463c33ef374dbf8f06eddba7a59390)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:16.683146 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b4ccc-15a5-430e-85ef-018139b48f88 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:16.697600 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ceb232c1-441a-4565-b5f4-ca3756aa065e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36ae74977a314572ac90f8bfa434332b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:16.786325 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ceb232c1-441a-4565-b5f4-ca3756aa065e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36ae74977a314572ac90f8bfa434332b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:16.786944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ceb232c1-441a-4565-b5f4-ca3756aa065e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36ae74977a314572ac90f8bfa434332b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:16.794289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ceb232c1-441a-4565-b5f4-ca3756aa065e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36ae74977a314572ac90f8bfa434332b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:16.812899 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:16.813313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-857b4ccc-15a5-430e-85ef-018139b48f88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa463c33ef374dbf8f06eddba7a59390', 'request_id': 'req-857b4ccc-15a5-430e-85ef-018139b48f88'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:16.814873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-ceb232c1-441a-4565-b5f4-ca3756aa065e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36ae74977a314572ac90f8bfa434332b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:16.819843 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-857b4ccc-15a5-430e-85ef-018139b48f88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa463c33ef374dbf8f06eddba7a59390', 'request_id': 'req-857b4ccc-15a5-430e-85ef-018139b48f88'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:16.821106 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1373/2714] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:32:16 2022] DELETE /identity/v3/users/36ae74977a314572ac90f8bfa434332b => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:32:16.828234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-857b4ccc-15a5-430e-85ef-018139b48f88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa463c33ef374dbf8f06eddba7a59390', 'request_id': 'req-857b4ccc-15a5-430e-85ef-018139b48f88'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:16.912808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-857b4ccc-15a5-430e-85ef-018139b48f88 admin admin] The token cache is being invalidate because project fa463c33ef374dbf8f06eddba7a59390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:32:16.913102 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-857b4ccc-15a5-430e-85ef-018139b48f88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:16.913739 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-857b4ccc-15a5-430e-85ef-018139b48f88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:16.915217 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1342/2715] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:32:16 2022] DELETE /identity/v3/projects/fa463c33ef374dbf8f06eddba7a59390 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:16.919295 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb232c1-441a-4565-b5f4-ca3756aa065e admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:16.920871 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ceb232c1-441a-4565-b5f4-ca3756aa065e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:16.985325 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb232c1-441a-4565-b5f4-ca3756aa065e admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:16.990584 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338a677-78d3-436b-888b-ea3b81b448d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:16.991203 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338a677-78d3-436b-888b-ea3b81b448d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:16.991485 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338a677-78d3-436b-888b-ea3b81b448d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:16.991759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338a677-78d3-436b-888b-ea3b81b448d6 admin admin] PATH_INFO: `/v3/users/dfe3a8aafc3c4d1ebefe98cd67fb2072` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:16.992907 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338a677-78d3-436b-888b-ea3b81b448d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfe3a8aafc3c4d1ebefe98cd67fb2072)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:16.994844 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338a677-78d3-436b-888b-ea3b81b448d6 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:17.182430 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c338a677-78d3-436b-888b-ea3b81b448d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfe3a8aafc3c4d1ebefe98cd67fb2072'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:17.256136 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b4ccc-15a5-430e-85ef-018139b48f88 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:17.257591 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-857b4ccc-15a5-430e-85ef-018139b48f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:17.286571 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c338a677-78d3-436b-888b-ea3b81b448d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfe3a8aafc3c4d1ebefe98cd67fb2072'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:17.287451 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c338a677-78d3-436b-888b-ea3b81b448d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfe3a8aafc3c4d1ebefe98cd67fb2072'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:17.295836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c338a677-78d3-436b-888b-ea3b81b448d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfe3a8aafc3c4d1ebefe98cd67fb2072'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:17.309475 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c338a677-78d3-436b-888b-ea3b81b448d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfe3a8aafc3c4d1ebefe98cd67fb2072'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:17.314558 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:17.316579 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1374/2716] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:32:16 2022] DELETE /identity/v3/users/dfe3a8aafc3c4d1ebefe98cd67fb2072 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:17.338995 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b4ccc-15a5-430e-85ef-018139b48f88 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:17.396036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d4b9c-a686-4d47-9af5-3d4e55a39511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:17.396709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d4b9c-a686-4d47-9af5-3d4e55a39511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:17.396998 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d4b9c-a686-4d47-9af5-3d4e55a39511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:17.397345 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d4b9c-a686-4d47-9af5-3d4e55a39511 admin admin] PATH_INFO: `/v3/projects/f548b356076c43e7a71df329c586ca8e` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:17.398468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d4b9c-a686-4d47-9af5-3d4e55a39511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f548b356076c43e7a71df329c586ca8e)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:17.400355 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d4b9c-a686-4d47-9af5-3d4e55a39511 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:17.526131 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-342d4b9c-a686-4d47-9af5-3d4e55a39511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f548b356076c43e7a71df329c586ca8e', 'request_id': 'req-342d4b9c-a686-4d47-9af5-3d4e55a39511'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:17.549969 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-342d4b9c-a686-4d47-9af5-3d4e55a39511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f548b356076c43e7a71df329c586ca8e', 'request_id': 'req-342d4b9c-a686-4d47-9af5-3d4e55a39511'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:17.560465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-342d4b9c-a686-4d47-9af5-3d4e55a39511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f548b356076c43e7a71df329c586ca8e', 'request_id': 'req-342d4b9c-a686-4d47-9af5-3d4e55a39511'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:17.632566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-342d4b9c-a686-4d47-9af5-3d4e55a39511 admin admin] The token cache is being invalidate because project f548b356076c43e7a71df329c586ca8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:32:17.632901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-342d4b9c-a686-4d47-9af5-3d4e55a39511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:17.633547 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-342d4b9c-a686-4d47-9af5-3d4e55a39511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:17.635477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1343/2717] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:32:17 2022] DELETE /identity/v3/projects/f548b356076c43e7a71df329c586ca8e => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:32:17.639935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1977ea-272d-4bdc-bdc1-cf3e7987dc87 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:17.640232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1977ea-272d-4bdc-bdc1-cf3e7987dc87 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:17.640497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1977ea-272d-4bdc-bdc1-cf3e7987dc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:17.662139 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-8d1977ea-272d-4bdc-bdc1-cf3e7987dc87 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:17.663365 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-8d1977ea-272d-4bdc-bdc1-cf3e7987dc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:17.725382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1375/2718] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:17.733263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f6e19-0ea6-46a1-8296-22628dc3b40c None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:17.733550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f6e19-0ea6-46a1-8296-22628dc3b40c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:17.733873 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f6e19-0ea6-46a1-8296-22628dc3b40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:17.753876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-c03f6e19-0ea6-46a1-8296-22628dc3b40c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:17.755098 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c03f6e19-0ea6-46a1-8296-22628dc3b40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:17.762822 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1344/2719] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:17.766966 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1977ea-272d-4bdc-bdc1-cf3e7987dc87 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:17.777884 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1977ea-272d-4bdc-bdc1-cf3e7987dc87 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:17.782535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5ce13-f404-4438-aecc-449c32f84b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:17.783162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5ce13-f404-4438-aecc-449c32f84b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:17.783442 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5ce13-f404-4438-aecc-449c32f84b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:17.783708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5ce13-f404-4438-aecc-449c32f84b0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:17.784514 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5ce13-f404-4438-aecc-449c32f84b0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:32:17.784874 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5ce13-f404-4438-aecc-449c32f84b0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:17.789352 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5ce13-f404-4438-aecc-449c32f84b0f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:17.798027 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1376/2720] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:32:17.803747 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f6e19-0ea6-46a1-8296-22628dc3b40c None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:17.813457 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f6e19-0ea6-46a1-8296-22628dc3b40c None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:17.818886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1329304-777b-4cf8-8946-319ecc746528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:17.819540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1329304-777b-4cf8-8946-319ecc746528 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:17.822891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1329304-777b-4cf8-8946-319ecc746528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:17.829567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1329304-777b-4cf8-8946-319ecc746528 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:17.829567 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1329304-777b-4cf8-8946-319ecc746528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:17.830219 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1329304-777b-4cf8-8946-319ecc746528 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:17.834065 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5ce13-f404-4438-aecc-449c32f84b0f admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:17.837495 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-6cc5ce13-f404-4438-aecc-449c32f84b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:17.846573 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:17.852163 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5ce13-f404-4438-aecc-449c32f84b0f admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:17.857258 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193c3f6-5054-4a9d-9749-1136d4780f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:17.858066 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193c3f6-5054-4a9d-9749-1136d4780f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:17.858455 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193c3f6-5054-4a9d-9749-1136d4780f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:17.858854 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193c3f6-5054-4a9d-9749-1136d4780f0f admin admin] PATH_INFO: `/v3/projects/eddc0596aa594795ab42424ada9a2267` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:17.860261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193c3f6-5054-4a9d-9749-1136d4780f0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=eddc0596aa594795ab42424ada9a2267)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:17.862335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193c3f6-5054-4a9d-9749-1136d4780f0f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:17.966399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1345/2721] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:32:17 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:32:17.970356 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1329304-777b-4cf8-8946-319ecc746528 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:18.019310 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1193c3f6-5054-4a9d-9749-1136d4780f0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eddc0596aa594795ab42424ada9a2267', 'request_id': 'req-1193c3f6-5054-4a9d-9749-1136d4780f0f'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:18.025669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1193c3f6-5054-4a9d-9749-1136d4780f0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eddc0596aa594795ab42424ada9a2267', 'request_id': 'req-1193c3f6-5054-4a9d-9749-1136d4780f0f'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:18.032920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1193c3f6-5054-4a9d-9749-1136d4780f0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eddc0596aa594795ab42424ada9a2267', 'request_id': 'req-1193c3f6-5054-4a9d-9749-1136d4780f0f'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:18.041020 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1329304-777b-4cf8-8946-319ecc746528 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:18.099847 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40e7e5-c3ed-4d67-b9bb-e17b8327f1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:18.100481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40e7e5-c3ed-4d67-b9bb-e17b8327f1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.100797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40e7e5-c3ed-4d67-b9bb-e17b8327f1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.101076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40e7e5-c3ed-4d67-b9bb-e17b8327f1e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.101735 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40e7e5-c3ed-4d67-b9bb-e17b8327f1e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:18.103607 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40e7e5-c3ed-4d67-b9bb-e17b8327f1e5 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:18.135318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1193c3f6-5054-4a9d-9749-1136d4780f0f admin admin] The token cache is being invalidate because project eddc0596aa594795ab42424ada9a2267 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:32:18.135697 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1193c3f6-5054-4a9d-9749-1136d4780f0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:18.136369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1193c3f6-5054-4a9d-9749-1136d4780f0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:18.137918 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1377/2722] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:32:17 2022] DELETE /identity/v3/projects/eddc0596aa594795ab42424ada9a2267 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:18.145553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b58265-a15f-424a-94e9-573add3ea282 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.145927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b58265-a15f-424a-94e9-573add3ea282 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.146206 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b58265-a15f-424a-94e9-573add3ea282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.165851 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-92b58265-a15f-424a-94e9-573add3ea282 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:18.167045 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-92b58265-a15f-424a-94e9-573add3ea282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:18.174810 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1378/2723] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:32:18.183907 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f7c48-0a14-4ddd-97e4-0393c524b1b6 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.184190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f7c48-0a14-4ddd-97e4-0393c524b1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.184466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f7c48-0a14-4ddd-97e4-0393c524b1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.203590 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-c02f7c48-0a14-4ddd-97e4-0393c524b1b6 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:18.206951 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1346/2724] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:32:17 2022] POST /identity/v3/users => generated 287 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:32:18.207081 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-c02f7c48-0a14-4ddd-97e4-0393c524b1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:18.208003 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40e7e5-c3ed-4d67-b9bb-e17b8327f1e5 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:18.209597 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-7c40e7e5-c3ed-4d67-b9bb-e17b8327f1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:18.213032 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1379/2725] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:32:18.216863 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f7c48-0a14-4ddd-97e4-0393c524b1b6 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:18.223021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40e7e5-c3ed-4d67-b9bb-e17b8327f1e5 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:18.226550 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f7c48-0a14-4ddd-97e4-0393c524b1b6 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:18.228186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323350c-d6d4-4067-989d-20b78205449a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:18.228994 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323350c-d6d4-4067-989d-20b78205449a admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.233112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323350c-d6d4-4067-989d-20b78205449a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.233112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323350c-d6d4-4067-989d-20b78205449a admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.233112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323350c-d6d4-4067-989d-20b78205449a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:18.233465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6923c11-22b3-46d3-8592-bc8e68d8621d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:18.233465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6923c11-22b3-46d3-8592-bc8e68d8621d admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.233684 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323350c-d6d4-4067-989d-20b78205449a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:18.233781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6923c11-22b3-46d3-8592-bc8e68d8621d admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.233781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6923c11-22b3-46d3-8592-bc8e68d8621d admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.233781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6923c11-22b3-46d3-8592-bc8e68d8621d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:32:18.233781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6923c11-22b3-46d3-8592-bc8e68d8621d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:18.234717 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6923c11-22b3-46d3-8592-bc8e68d8621d admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:18.245979 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1347/2726] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:32:18.246974 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323350c-d6d4-4067-989d-20b78205449a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:18.247839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1380/2727] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 12 12:32:18.251848 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6923c11-22b3-46d3-8592-bc8e68d8621d admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:18.257530 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323350c-d6d4-4067-989d-20b78205449a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:18.262070 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6923c11-22b3-46d3-8592-bc8e68d8621d admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:18.262585 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3f921-3fe8-4927-b371-cf79d8c8f8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:18.263210 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3f921-3fe8-4927-b371-cf79d8c8f8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.263541 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3f921-3fe8-4927-b371-cf79d8c8f8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.263819 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3f921-3fe8-4927-b371-cf79d8c8f8b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.264497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3f921-3fe8-4927-b371-cf79d8c8f8b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:18.266234 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3f921-3fe8-4927-b371-cf79d8c8f8b0 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:18.267312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c7be2-4f4d-4503-a97f-005803a4993f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:18.267894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c7be2-4f4d-4503-a97f-005803a4993f admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.268174 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c7be2-4f4d-4503-a97f-005803a4993f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.268444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c7be2-4f4d-4503-a97f-005803a4993f admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.269113 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c7be2-4f4d-4503-a97f-005803a4993f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:18.270998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c7be2-4f4d-4503-a97f-005803a4993f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:18.277402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1348/2728] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:32:18.281431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3f921-3fe8-4927-b371-cf79d8c8f8b0 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:18.291679 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3f921-3fe8-4927-b371-cf79d8c8f8b0 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:18.296306 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f2910-3e00-4e85-a895-6f5b9f5e86ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:18.296953 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f2910-3e00-4e85-a895-6f5b9f5e86ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.297235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f2910-3e00-4e85-a895-6f5b9f5e86ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.297513 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f2910-3e00-4e85-a895-6f5b9f5e86ea admin admin] PATH_INFO: `/v3/projects/5dded162e47d494d9683568544d446bb/users/910098074b83416fbf14664ef99d9227/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.312560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f2910-3e00-4e85-a895-6f5b9f5e86ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dded162e47d494d9683568544d446bb, user_id=910098074b83416fbf14664ef99d9227, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:18.315180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f2910-3e00-4e85-a895-6f5b9f5e86ea admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:18.381803 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1381/2729] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:32:18 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:32:18.386037 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c7be2-4f4d-4503-a97f-005803a4993f admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:18.404552 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1349/2730] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:18 2022] PUT /identity/v3/projects/5dded162e47d494d9683568544d446bb/users/910098074b83416fbf14664ef99d9227/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:32:18.411836 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae22df-8a35-4bce-8623-782ddbe0b417 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.412118 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae22df-8a35-4bce-8623-782ddbe0b417 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.412405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae22df-8a35-4bce-8623-782ddbe0b417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.421492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:18.434106 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-2fae22df-8a35-4bce-8623-782ddbe0b417 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:18.435514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2fae22df-8a35-4bce-8623-782ddbe0b417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:18.450868 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c7be2-4f4d-4503-a97f-005803a4993f admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:18.496764 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1350/2731] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:32:18.506732 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae22df-8a35-4bce-8623-782ddbe0b417 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:18.508065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2fae22df-8a35-4bce-8623-782ddbe0b417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:18.508739 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd9e57-9d65-4133-9db8-45ae3d4bbf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:18.509355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd9e57-9d65-4133-9db8-45ae3d4bbf56 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.513088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd9e57-9d65-4133-9db8-45ae3d4bbf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.513088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd9e57-9d65-4133-9db8-45ae3d4bbf56 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.513088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd9e57-9d65-4133-9db8-45ae3d4bbf56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:18.513088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd9e57-9d65-4133-9db8-45ae3d4bbf56 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:18.567191 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae22df-8a35-4bce-8623-782ddbe0b417 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:18.571005 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82620a51-6d70-4d10-9081-3be54e882e74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:18.571620 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82620a51-6d70-4d10-9081-3be54e882e74 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.571904 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82620a51-6d70-4d10-9081-3be54e882e74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.572179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82620a51-6d70-4d10-9081-3be54e882e74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.583141 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82620a51-6d70-4d10-9081-3be54e882e74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:18.593103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82620a51-6d70-4d10-9081-3be54e882e74 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:18.593237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1382/2732] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:32:18 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:32:18.593237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd9e57-9d65-4133-9db8-45ae3d4bbf56 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:18.595404 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1351/2733] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:32:18.603394 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd9e57-9d65-4133-9db8-45ae3d4bbf56 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:18.608076 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3057fa-5531-4e82-a67d-7fa82b71b957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:18.608710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3057fa-5531-4e82-a67d-7fa82b71b957 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.608999 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3057fa-5531-4e82-a67d-7fa82b71b957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.609278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3057fa-5531-4e82-a67d-7fa82b71b957 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.609963 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3057fa-5531-4e82-a67d-7fa82b71b957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:18.611700 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3057fa-5531-4e82-a67d-7fa82b71b957 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:18.622392 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1383/2734] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:32:18.626600 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82620a51-6d70-4d10-9081-3be54e882e74 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:18.636444 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82620a51-6d70-4d10-9081-3be54e882e74 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:18.641143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd91a8-33ff-4523-86f3-6f1a93603ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:18.641786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd91a8-33ff-4523-86f3-6f1a93603ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.642145 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd91a8-33ff-4523-86f3-6f1a93603ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.642461 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd91a8-33ff-4523-86f3-6f1a93603ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.643183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd91a8-33ff-4523-86f3-6f1a93603ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:18.645535 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd91a8-33ff-4523-86f3-6f1a93603ff8 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:18.655332 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:18.655626 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1352/2735] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:32:18.660477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3057fa-5531-4e82-a67d-7fa82b71b957 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:18.670426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3057fa-5531-4e82-a67d-7fa82b71b957 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:18.675157 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566820e3-4957-4283-90e9-687768c30f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:18.675781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566820e3-4957-4283-90e9-687768c30f8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.676067 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566820e3-4957-4283-90e9-687768c30f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.676414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566820e3-4957-4283-90e9-687768c30f8a admin admin] PATH_INFO: `/v3/projects/bf558c56556a4fad8945d1f5f59a1916/users/84674d12c30e4c71ab47f63a20d9739c/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.691365 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566820e3-4957-4283-90e9-687768c30f8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf558c56556a4fad8945d1f5f59a1916, user_id=84674d12c30e4c71ab47f63a20d9739c, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:18.694150 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566820e3-4957-4283-90e9-687768c30f8a admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:18.815377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1384/2736] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:18 2022] PUT /identity/v3/projects/bf558c56556a4fad8945d1f5f59a1916/users/84674d12c30e4c71ab47f63a20d9739c/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:18.825630 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0925f-7773-4448-a920-7ed8f3abe797 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.826178 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0925f-7773-4448-a920-7ed8f3abe797 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.826666 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0925f-7773-4448-a920-7ed8f3abe797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.855022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-59e0925f-7773-4448-a920-7ed8f3abe797 None None] MFA Rules not processed for user `84674d12c30e4c71ab47f63a20d9739c`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:18.856214 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-59e0925f-7773-4448-a920-7ed8f3abe797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:18.922746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1353/2737] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:18.927349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566820e3-4957-4283-90e9-687768c30f8a admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:18.990923 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566820e3-4957-4283-90e9-687768c30f8a admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:18.995547 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc47b1f-bec8-4524-973b-07fcd5c8cadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:18.996141 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc47b1f-bec8-4524-973b-07fcd5c8cadc admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:18.996450 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc47b1f-bec8-4524-973b-07fcd5c8cadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:18.996767 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc47b1f-bec8-4524-973b-07fcd5c8cadc admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:18.997393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc47b1f-bec8-4524-973b-07fcd5c8cadc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:18.999327 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc47b1f-bec8-4524-973b-07fcd5c8cadc admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:19.009812 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:19.137023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1385/2738] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:32:18 2022] POST /identity/v3/projects => generated 387 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:32:19.141075 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0925f-7773-4448-a920-7ed8f3abe797 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:19.154083 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0925f-7773-4448-a920-7ed8f3abe797 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:19.221239 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084eec12-9a6b-47e7-ba95-c5809f005e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:19.222461 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084eec12-9a6b-47e7-ba95-c5809f005e64 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:19.222902 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084eec12-9a6b-47e7-ba95-c5809f005e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:19.223186 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084eec12-9a6b-47e7-ba95-c5809f005e64 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:19.223817 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084eec12-9a6b-47e7-ba95-c5809f005e64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:19.225909 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084eec12-9a6b-47e7-ba95-c5809f005e64 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:19.421931 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1354/2739] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:32:19 2022] POST /identity/v3/users => generated 300 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:32:19.426145 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc47b1f-bec8-4524-973b-07fcd5c8cadc admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:19.438484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc47b1f-bec8-4524-973b-07fcd5c8cadc admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:19.443374 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e65f61-afe8-4bab-8ff4-804d9f0553f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:19.443990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e65f61-afe8-4bab-8ff4-804d9f0553f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:19.444281 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e65f61-afe8-4bab-8ff4-804d9f0553f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:19.444562 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e65f61-afe8-4bab-8ff4-804d9f0553f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:19.445266 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e65f61-afe8-4bab-8ff4-804d9f0553f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:19.446869 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e65f61-afe8-4bab-8ff4-804d9f0553f1 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:19.458852 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1386/2740] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:32:19.463322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084eec12-9a6b-47e7-ba95-c5809f005e64 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:19.474088 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084eec12-9a6b-47e7-ba95-c5809f005e64 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:19.478841 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed0b62-1c39-4847-9508-78abda7ef065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:19.479560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed0b62-1c39-4847-9508-78abda7ef065 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:19.479901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed0b62-1c39-4847-9508-78abda7ef065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:19.480196 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed0b62-1c39-4847-9508-78abda7ef065 admin admin] PATH_INFO: `/v3/projects/0c025040daec480b9498eaea7a490429/users/76038d74b28944649ae878d60edd4717/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:19.495333 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed0b62-1c39-4847-9508-78abda7ef065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c025040daec480b9498eaea7a490429, user_id=76038d74b28944649ae878d60edd4717, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:19.497833 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed0b62-1c39-4847-9508-78abda7ef065 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:19.587137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1355/2741] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:19 2022] PUT /identity/v3/projects/0c025040daec480b9498eaea7a490429/users/76038d74b28944649ae878d60edd4717/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:19.593608 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e65f61-afe8-4bab-8ff4-804d9f0553f1 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:19.605059 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e65f61-afe8-4bab-8ff4-804d9f0553f1 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:19.662036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285fbedc-cd31-488b-b92d-6a6c92411555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:19.662647 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fbedc-cd31-488b-b92d-6a6c92411555 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:19.662928 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fbedc-cd31-488b-b92d-6a6c92411555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:19.663209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fbedc-cd31-488b-b92d-6a6c92411555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:19.663850 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285fbedc-cd31-488b-b92d-6a6c92411555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:19.665490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285fbedc-cd31-488b-b92d-6a6c92411555 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:19.679018 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1387/2742] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:32:19.680299 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed0b62-1c39-4847-9508-78abda7ef065 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:19.692050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed0b62-1c39-4847-9508-78abda7ef065 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:19.696829 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45143d-05e3-487d-94d6-ba9d998b43d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:19.697519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45143d-05e3-487d-94d6-ba9d998b43d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:19.697813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45143d-05e3-487d-94d6-ba9d998b43d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:19.698090 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45143d-05e3-487d-94d6-ba9d998b43d7 admin admin] PATH_INFO: `/v3/projects/0c025040daec480b9498eaea7a490429/users/76038d74b28944649ae878d60edd4717/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:19.700081 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45143d-05e3-487d-94d6-ba9d998b43d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c025040daec480b9498eaea7a490429, user_id=76038d74b28944649ae878d60edd4717, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:19.702784 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45143d-05e3-487d-94d6-ba9d998b43d7 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:19.715166 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1356/2743] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:19 2022] PUT /identity/v3/projects/0c025040daec480b9498eaea7a490429/users/76038d74b28944649ae878d60edd4717/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:19.726780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e20db3-5ea9-4a6d-b5b6-2422c8c79a15 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:19.727069 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e20db3-5ea9-4a6d-b5b6-2422c8c79a15 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:19.727390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e20db3-5ea9-4a6d-b5b6-2422c8c79a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:19.747694 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:19.757492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-76e20db3-5ea9-4a6d-b5b6-2422c8c79a15 None None] MFA Rules not processed for user `76038d74b28944649ae878d60edd4717`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:19.758706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-76e20db3-5ea9-4a6d-b5b6-2422c8c79a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:19.825247 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1388/2744] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:19.836706 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45143d-05e3-487d-94d6-ba9d998b43d7 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:19.838373 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2f45143d-05e3-487d-94d6-ba9d998b43d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:19.858616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:19.898301 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45143d-05e3-487d-94d6-ba9d998b43d7 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:19.902142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef01347-5c9d-4fac-8cbd-c7ecf4a14738 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:19.902745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef01347-5c9d-4fac-8cbd-c7ecf4a14738 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:19.903028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef01347-5c9d-4fac-8cbd-c7ecf4a14738 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:19.903300 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef01347-5c9d-4fac-8cbd-c7ecf4a14738 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:19.912720 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef01347-5c9d-4fac-8cbd-c7ecf4a14738 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:19.914123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef01347-5c9d-4fac-8cbd-c7ecf4a14738 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:19.923678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1357/2745] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:32:20.072761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e20db3-5ea9-4a6d-b5b6-2422c8c79a15 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:20.156983 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e20db3-5ea9-4a6d-b5b6-2422c8c79a15 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:20.163217 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d3db9-fb10-4340-9aee-ceef774fa799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:20.164078 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d3db9-fb10-4340-9aee-ceef774fa799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:20.164528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d3db9-fb10-4340-9aee-ceef774fa799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:20.164928 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d3db9-fb10-4340-9aee-ceef774fa799 admin admin] PATH_INFO: `/v3/users/84674d12c30e4c71ab47f63a20d9739c` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:20.166312 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d3db9-fb10-4340-9aee-ceef774fa799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84674d12c30e4c71ab47f63a20d9739c)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:20.168612 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d3db9-fb10-4340-9aee-ceef774fa799 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:20.188711 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec6c97-3aa9-4d9c-95da-1a5395d186f0 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:20.189174 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec6c97-3aa9-4d9c-95da-1a5395d186f0 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:20.190106 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec6c97-3aa9-4d9c-95da-1a5395d186f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:20.221888 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-8cec6c97-3aa9-4d9c-95da-1a5395d186f0 None None] MFA Rules not processed for user `910098074b83416fbf14664ef99d9227`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:20.223282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8cec6c97-3aa9-4d9c-95da-1a5395d186f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:20.289577 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1358/2746] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:32:20.293920 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec6c97-3aa9-4d9c-95da-1a5395d186f0 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:20.305482 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec6c97-3aa9-4d9c-95da-1a5395d186f0 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:20.310025 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828efbe-a6dd-465f-ab4a-37a8e2bd12db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:20.310603 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828efbe-a6dd-465f-ab4a-37a8e2bd12db admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:20.310884 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828efbe-a6dd-465f-ab4a-37a8e2bd12db admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:20.311153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828efbe-a6dd-465f-ab4a-37a8e2bd12db admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:20.311761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828efbe-a6dd-465f-ab4a-37a8e2bd12db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:20.313669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828efbe-a6dd-465f-ab4a-37a8e2bd12db admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:20.404709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1359/2747] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:32:20 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:32:20.409014 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828efbe-a6dd-465f-ab4a-37a8e2bd12db admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:20.442807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1a9d3db9-fb10-4340-9aee-ceef774fa799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84674d12c30e4c71ab47f63a20d9739c'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:20.471018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828efbe-a6dd-465f-ab4a-37a8e2bd12db admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:20.476180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570502e-0486-410b-b50a-e40d2ac684ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:20.476815 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570502e-0486-410b-b50a-e40d2ac684ed admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:20.477099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570502e-0486-410b-b50a-e40d2ac684ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:20.477375 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570502e-0486-410b-b50a-e40d2ac684ed admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:20.478022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570502e-0486-410b-b50a-e40d2ac684ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:20.479926 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570502e-0486-410b-b50a-e40d2ac684ed admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:20.517089 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1a9d3db9-fb10-4340-9aee-ceef774fa799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84674d12c30e4c71ab47f63a20d9739c'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:20.517737 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1a9d3db9-fb10-4340-9aee-ceef774fa799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84674d12c30e4c71ab47f63a20d9739c'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:20.525105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1a9d3db9-fb10-4340-9aee-ceef774fa799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84674d12c30e4c71ab47f63a20d9739c'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:20.537487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-1a9d3db9-fb10-4340-9aee-ceef774fa799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84674d12c30e4c71ab47f63a20d9739c'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:20.543742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1389/2748] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:32:20 2022] DELETE /identity/v3/users/84674d12c30e4c71ab47f63a20d9739c => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:20.547922 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4664e2-b713-485a-aab1-7595a81744f8 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:20.548213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4664e2-b713-485a-aab1-7595a81744f8 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:20.548488 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4664e2-b713-485a-aab1-7595a81744f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:20.567557 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-0c4664e2-b713-485a-aab1-7595a81744f8 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:20.568797 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0c4664e2-b713-485a-aab1-7595a81744f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:20.576899 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:20.621381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1360/2749] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:32:20 2022] POST /identity/v3/users => generated 292 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:32:20.625256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570502e-0486-410b-b50a-e40d2ac684ed admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:20.626923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-7570502e-0486-410b-b50a-e40d2ac684ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:20.627069 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1390/2750] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:32:20.638369 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4664e2-b713-485a-aab1-7595a81744f8 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:20.639591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-0c4664e2-b713-485a-aab1-7595a81744f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:20.641975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570502e-0486-410b-b50a-e40d2ac684ed admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:20.646783 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e0fd1-cf23-45bb-8c0a-0d8533364106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:20.647902 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e0fd1-cf23-45bb-8c0a-0d8533364106 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:20.647902 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e0fd1-cf23-45bb-8c0a-0d8533364106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:20.647902 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e0fd1-cf23-45bb-8c0a-0d8533364106 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:20.648659 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e0fd1-cf23-45bb-8c0a-0d8533364106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:20.651614 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e0fd1-cf23-45bb-8c0a-0d8533364106 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:20.669466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1361/2751] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:32:20.674454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e0fd1-cf23-45bb-8c0a-0d8533364106 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:20.688374 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:20.688374 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e0fd1-cf23-45bb-8c0a-0d8533364106 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:20.690787 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30f025-3325-416b-abea-faabd3f22e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:20.691564 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30f025-3325-416b-abea-faabd3f22e0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:20.691935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30f025-3325-416b-abea-faabd3f22e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:20.692226 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30f025-3325-416b-abea-faabd3f22e0d admin admin] PATH_INFO: `/v3/projects/8da1de6892394064a0f3b8cf171b9bbc/users/311ae2a7af7b464e898432619ed3f449/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:20.702484 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4664e2-b713-485a-aab1-7595a81744f8 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:20.705633 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05182ef-3d93-4d5b-a82a-9b34a8739afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:20.706232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05182ef-3d93-4d5b-a82a-9b34a8739afd service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:20.706515 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05182ef-3d93-4d5b-a82a-9b34a8739afd service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:20.706804 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05182ef-3d93-4d5b-a82a-9b34a8739afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:20.708082 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30f025-3325-416b-abea-faabd3f22e0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8da1de6892394064a0f3b8cf171b9bbc, user_id=311ae2a7af7b464e898432619ed3f449, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:20.711066 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30f025-3325-416b-abea-faabd3f22e0d admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:20.716820 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05182ef-3d93-4d5b-a82a-9b34a8739afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:20.717965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05182ef-3d93-4d5b-a82a-9b34a8739afd service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:20.728815 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1391/2752] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:32:20.819795 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1362/2753] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:20 2022] PUT /identity/v3/projects/8da1de6892394064a0f3b8cf171b9bbc/users/311ae2a7af7b464e898432619ed3f449/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:20.823614 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05182ef-3d93-4d5b-a82a-9b34a8739afd service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:20.833586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05182ef-3d93-4d5b-a82a-9b34a8739afd service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:20.888088 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f188692-fd88-4c6e-b358-efd5b64cb083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:20.888708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f188692-fd88-4c6e-b358-efd5b64cb083 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:20.888990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f188692-fd88-4c6e-b358-efd5b64cb083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:20.889260 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f188692-fd88-4c6e-b358-efd5b64cb083 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:20.889896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f188692-fd88-4c6e-b358-efd5b64cb083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:20.891522 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f188692-fd88-4c6e-b358-efd5b64cb083 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:20.901658 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1392/2754] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:32:20.905805 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30f025-3325-416b-abea-faabd3f22e0d admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:20.917032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30f025-3325-416b-abea-faabd3f22e0d admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:20.921669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45592036-362d-42e3-9492-042dee83e6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:20.922332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45592036-362d-42e3-9492-042dee83e6b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:20.922611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45592036-362d-42e3-9492-042dee83e6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:20.922885 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45592036-362d-42e3-9492-042dee83e6b4 admin admin] PATH_INFO: `/v3/projects/8da1de6892394064a0f3b8cf171b9bbc/users/311ae2a7af7b464e898432619ed3f449/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:20.924983 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45592036-362d-42e3-9492-042dee83e6b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8da1de6892394064a0f3b8cf171b9bbc, user_id=311ae2a7af7b464e898432619ed3f449, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:20.927786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45592036-362d-42e3-9492-042dee83e6b4 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:20.937826 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1363/2755] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:20 2022] PUT /identity/v3/projects/8da1de6892394064a0f3b8cf171b9bbc/users/311ae2a7af7b464e898432619ed3f449/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:21.233248 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f188692-fd88-4c6e-b358-efd5b64cb083 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:21.235096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-7f188692-fd88-4c6e-b358-efd5b64cb083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:21.254006 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f188692-fd88-4c6e-b358-efd5b64cb083 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:21.304960 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:21.313552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7772120-fae9-4764-b20f-7dfd9373ff16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:21.314182 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7772120-fae9-4764-b20f-7dfd9373ff16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:21.314466 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7772120-fae9-4764-b20f-7dfd9373ff16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:21.314742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7772120-fae9-4764-b20f-7dfd9373ff16 admin admin] PATH_INFO: `/v3/projects/bf558c56556a4fad8945d1f5f59a1916` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:21.316495 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7772120-fae9-4764-b20f-7dfd9373ff16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf558c56556a4fad8945d1f5f59a1916)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:21.318622 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7772120-fae9-4764-b20f-7dfd9373ff16 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:21.439328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f7772120-fae9-4764-b20f-7dfd9373ff16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf558c56556a4fad8945d1f5f59a1916', 'request_id': 'req-f7772120-fae9-4764-b20f-7dfd9373ff16'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:21.445505 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f7772120-fae9-4764-b20f-7dfd9373ff16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf558c56556a4fad8945d1f5f59a1916', 'request_id': 'req-f7772120-fae9-4764-b20f-7dfd9373ff16'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:21.451940 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f7772120-fae9-4764-b20f-7dfd9373ff16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf558c56556a4fad8945d1f5f59a1916', 'request_id': 'req-f7772120-fae9-4764-b20f-7dfd9373ff16'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:21.533583 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f7772120-fae9-4764-b20f-7dfd9373ff16 admin admin] The token cache is being invalidate because project bf558c56556a4fad8945d1f5f59a1916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:32:21.533908 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f7772120-fae9-4764-b20f-7dfd9373ff16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:21.534390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f7772120-fae9-4764-b20f-7dfd9373ff16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:21.536063 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1393/2756] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:32:21 2022] DELETE /identity/v3/projects/bf558c56556a4fad8945d1f5f59a1916 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:21.539992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45592036-362d-42e3-9492-042dee83e6b4 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:21.541720 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-45592036-362d-42e3-9492-042dee83e6b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:21.554712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45592036-362d-42e3-9492-042dee83e6b4 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:21.615627 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd96bc-df02-4e31-ae4f-7a390050a4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:21.616276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd96bc-df02-4e31-ae4f-7a390050a4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:21.616557 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd96bc-df02-4e31-ae4f-7a390050a4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:21.616857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd96bc-df02-4e31-ae4f-7a390050a4cf admin admin] PATH_INFO: `/v3/users/76038d74b28944649ae878d60edd4717` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:21.617972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd96bc-df02-4e31-ae4f-7a390050a4cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=76038d74b28944649ae878d60edd4717)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:21.619944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd96bc-df02-4e31-ae4f-7a390050a4cf admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:21.874232 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-fbfd96bc-df02-4e31-ae4f-7a390050a4cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76038d74b28944649ae878d60edd4717'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:21.874888 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-fbfd96bc-df02-4e31-ae4f-7a390050a4cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76038d74b28944649ae878d60edd4717'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:21.946175 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-fbfd96bc-df02-4e31-ae4f-7a390050a4cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76038d74b28944649ae878d60edd4717'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:21.960478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-fbfd96bc-df02-4e31-ae4f-7a390050a4cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76038d74b28944649ae878d60edd4717'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:21.967909 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-fbfd96bc-df02-4e31-ae4f-7a390050a4cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76038d74b28944649ae878d60edd4717'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:21.974350 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1364/2757] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:32:21 2022] DELETE /identity/v3/users/76038d74b28944649ae878d60edd4717 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:22.460715 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7772120-fae9-4764-b20f-7dfd9373ff16 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:22.462923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edf917-28a4-40ba-bd2a-fe9424940111 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:22.463205 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edf917-28a4-40ba-bd2a-fe9424940111 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:22.463479 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edf917-28a4-40ba-bd2a-fe9424940111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:22.474843 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7772120-fae9-4764-b20f-7dfd9373ff16 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:22.507804 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-45edf917-28a4-40ba-bd2a-fe9424940111 None None] MFA Rules not processed for user `311ae2a7af7b464e898432619ed3f449`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:22.509744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-45edf917-28a4-40ba-bd2a-fe9424940111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:22.561432 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:22.573682 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d7676-c13b-44d6-a40a-ec1b3af52a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:22.574333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d7676-c13b-44d6-a40a-ec1b3af52a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:22.574617 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d7676-c13b-44d6-a40a-ec1b3af52a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:22.575493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d7676-c13b-44d6-a40a-ec1b3af52a98 admin admin] PATH_INFO: `/v3/projects/0c025040daec480b9498eaea7a490429` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:22.576703 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d7676-c13b-44d6-a40a-ec1b3af52a98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c025040daec480b9498eaea7a490429)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:22.578621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d7676-c13b-44d6-a40a-ec1b3af52a98 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:22.602183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1365/2758] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:22.614330 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edf917-28a4-40ba-bd2a-fe9424940111 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:22.615712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-45edf917-28a4-40ba-bd2a-fe9424940111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:22.673237 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edf917-28a4-40ba-bd2a-fe9424940111 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:22.673493 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-a72d7676-c13b-44d6-a40a-ec1b3af52a98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c025040daec480b9498eaea7a490429', 'request_id': 'req-a72d7676-c13b-44d6-a40a-ec1b3af52a98'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:22.679741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-a72d7676-c13b-44d6-a40a-ec1b3af52a98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c025040daec480b9498eaea7a490429', 'request_id': 'req-a72d7676-c13b-44d6-a40a-ec1b3af52a98'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:22.686601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-a72d7676-c13b-44d6-a40a-ec1b3af52a98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c025040daec480b9498eaea7a490429', 'request_id': 'req-a72d7676-c13b-44d6-a40a-ec1b3af52a98'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:22.724676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627112b-78d3-4bff-8661-fc89f5156c5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:22.725275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627112b-78d3-4bff-8661-fc89f5156c5a service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:22.725592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627112b-78d3-4bff-8661-fc89f5156c5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:22.725866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627112b-78d3-4bff-8661-fc89f5156c5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:22.727600 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d627112b-78d3-4bff-8661-fc89f5156c5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:22.745759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-a72d7676-c13b-44d6-a40a-ec1b3af52a98 admin admin] The token cache is being invalidate because project 0c025040daec480b9498eaea7a490429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:32:22.746049 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-a72d7676-c13b-44d6-a40a-ec1b3af52a98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:22.746702 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-a72d7676-c13b-44d6-a40a-ec1b3af52a98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:22.748490 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1394/2759] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:32:22 2022] DELETE /identity/v3/projects/0c025040daec480b9498eaea7a490429 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:22.753009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b9bc5-52bd-4447-9f08-bc0069cf4415 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:22.753293 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b9bc5-52bd-4447-9f08-bc0069cf4415 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:22.753570 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b9bc5-52bd-4447-9f08-bc0069cf4415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:22.774116 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-c87b9bc5-52bd-4447-9f08-bc0069cf4415 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:22.775335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-c87b9bc5-52bd-4447-9f08-bc0069cf4415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:22.784242 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:22.791439 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627112b-78d3-4bff-8661-fc89f5156c5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:22.794335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627112b-78d3-4bff-8661-fc89f5156c5a service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:22.802645 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1366/2760] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:32:22.850078 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1395/2761] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:32:22.857948 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b95d0-4d98-44b6-98a5-effeeef56b2e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:22.858311 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b95d0-4d98-44b6-98a5-effeeef56b2e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:22.858667 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b95d0-4d98-44b6-98a5-effeeef56b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:22.880294 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-0b9b95d0-4d98-44b6-98a5-effeeef56b2e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:22.881557 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0b9b95d0-4d98-44b6-98a5-effeeef56b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:22.890010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1367/2762] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:22.896568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b9bc5-52bd-4447-9f08-bc0069cf4415 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:22.906355 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b9bc5-52bd-4447-9f08-bc0069cf4415 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:22.911050 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cd54b-bc55-4c4c-ac82-73765857e102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:22.911644 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cd54b-bc55-4c4c-ac82-73765857e102 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:22.911926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cd54b-bc55-4c4c-ac82-73765857e102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:22.912200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cd54b-bc55-4c4c-ac82-73765857e102 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:22.912874 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cd54b-bc55-4c4c-ac82-73765857e102 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:32:22.913179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cd54b-bc55-4c4c-ac82-73765857e102 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:22.914781 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cd54b-bc55-4c4c-ac82-73765857e102 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:22.927038 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1396/2763] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:32:22.931122 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b95d0-4d98-44b6-98a5-effeeef56b2e None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:22.940472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b95d0-4d98-44b6-98a5-effeeef56b2e None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:22.945832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ad04d-6384-4144-80a2-5850b0580418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:22.946422 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ad04d-6384-4144-80a2-5850b0580418 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:22.946710 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ad04d-6384-4144-80a2-5850b0580418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:22.946990 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ad04d-6384-4144-80a2-5850b0580418 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:22.947592 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ad04d-6384-4144-80a2-5850b0580418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:22.949529 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ad04d-6384-4144-80a2-5850b0580418 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:23.055997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1368/2764] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:32:22 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:32:23.060511 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cd54b-bc55-4c4c-ac82-73765857e102 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:23.071326 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cd54b-bc55-4c4c-ac82-73765857e102 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:23.125955 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a686913-25be-4def-a258-d80a64ef7489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:23.126548 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a686913-25be-4def-a258-d80a64ef7489 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:23.126829 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a686913-25be-4def-a258-d80a64ef7489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:23.127103 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a686913-25be-4def-a258-d80a64ef7489 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:23.127712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a686913-25be-4def-a258-d80a64ef7489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:23.129929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a686913-25be-4def-a258-d80a64ef7489 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:23.237192 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1397/2765] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:32:23 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:32:23.241311 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ad04d-6384-4144-80a2-5850b0580418 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:23.251613 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ad04d-6384-4144-80a2-5850b0580418 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:23.256240 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e87fb-7ca3-4310-ac84-2847a088b056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:23.256872 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e87fb-7ca3-4310-ac84-2847a088b056 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:23.257152 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e87fb-7ca3-4310-ac84-2847a088b056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:23.257424 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e87fb-7ca3-4310-ac84-2847a088b056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:23.258065 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e87fb-7ca3-4310-ac84-2847a088b056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:23.259723 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e87fb-7ca3-4310-ac84-2847a088b056 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:23.270669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1369/2766] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:32:23.274628 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a686913-25be-4def-a258-d80a64ef7489 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:23.284986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a686913-25be-4def-a258-d80a64ef7489 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:23.290519 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79958f77-b164-4dc4-adde-89058c52d7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:23.291113 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79958f77-b164-4dc4-adde-89058c52d7d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:23.291396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79958f77-b164-4dc4-adde-89058c52d7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:23.291670 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79958f77-b164-4dc4-adde-89058c52d7d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:23.292309 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79958f77-b164-4dc4-adde-89058c52d7d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:23.294015 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79958f77-b164-4dc4-adde-89058c52d7d4 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:23.305604 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1398/2767] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:32:23.309696 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e87fb-7ca3-4310-ac84-2847a088b056 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:23.314770 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:23.320151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e87fb-7ca3-4310-ac84-2847a088b056 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:23.324997 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43048eca-ca4d-4040-bae9-cb84a0388e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:23.325608 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43048eca-ca4d-4040-bae9-cb84a0388e92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:23.325889 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43048eca-ca4d-4040-bae9-cb84a0388e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:23.326162 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43048eca-ca4d-4040-bae9-cb84a0388e92 admin admin] PATH_INFO: `/v3/projects/ae03acd916ee471a871dce2bccc399c7/users/33ae07492df24916ab68e37194b086f9/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:23.341425 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43048eca-ca4d-4040-bae9-cb84a0388e92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae03acd916ee471a871dce2bccc399c7, user_id=33ae07492df24916ab68e37194b086f9, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:23.344043 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43048eca-ca4d-4040-bae9-cb84a0388e92 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:23.408975 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1370/2768] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:23 2022] PUT /identity/v3/projects/ae03acd916ee471a871dce2bccc399c7/users/33ae07492df24916ab68e37194b086f9/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:23.419601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f960d-55ee-4774-a236-f227190c71c8 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:23.419887 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f960d-55ee-4774-a236-f227190c71c8 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:23.420162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f960d-55ee-4774-a236-f227190c71c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:23.455892 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-667f960d-55ee-4774-a236-f227190c71c8 None None] MFA Rules not processed for user `33ae07492df24916ab68e37194b086f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:23.457790 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-667f960d-55ee-4774-a236-f227190c71c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:23.524421 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1399/2769] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:23.537329 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43048eca-ca4d-4040-bae9-cb84a0388e92 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:23.537692 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-43048eca-ca4d-4040-bae9-cb84a0388e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:23.606589 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43048eca-ca4d-4040-bae9-cb84a0388e92 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:23.610376 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5cdc51-0eb7-4770-bf16-b247a2b6115e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:23.610983 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cdc51-0eb7-4770-bf16-b247a2b6115e service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:23.611270 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cdc51-0eb7-4770-bf16-b247a2b6115e service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:23.611546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cdc51-0eb7-4770-bf16-b247a2b6115e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:23.620446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5cdc51-0eb7-4770-bf16-b247a2b6115e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:23.622037 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5cdc51-0eb7-4770-bf16-b247a2b6115e service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:23.631773 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1371/2770] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:32:24.562095 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f960d-55ee-4774-a236-f227190c71c8 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:24.563513 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-667f960d-55ee-4774-a236-f227190c71c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:24.575452 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f960d-55ee-4774-a236-f227190c71c8 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:24.578996 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a3868-f6f4-445c-a178-fd2b86e74833 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:24.579586 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a3868-f6f4-445c-a178-fd2b86e74833 service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:24.579865 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a3868-f6f4-445c-a178-fd2b86e74833 service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:24.580132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a3868-f6f4-445c-a178-fd2b86e74833 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:24.582335 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-930a3868-f6f4-445c-a178-fd2b86e74833 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:24.588319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:24.642980 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a3868-f6f4-445c-a178-fd2b86e74833 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:24.645042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a3868-f6f4-445c-a178-fd2b86e74833 service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:24.654269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1400/2771] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:32:25.288688 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5cdc51-0eb7-4770-bf16-b247a2b6115e service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:25.297860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5cdc51-0eb7-4770-bf16-b247a2b6115e service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:25.302646 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2a194-c9f1-4f6a-811b-f4aa42813cb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:25.303435 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2a194-c9f1-4f6a-811b-f4aa42813cb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:25.308322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2a194-c9f1-4f6a-811b-f4aa42813cb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:25.308322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2a194-c9f1-4f6a-811b-f4aa42813cb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:25.308322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a4e2a194-c9f1-4f6a-811b-f4aa42813cb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:25.369285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2a194-c9f1-4f6a-811b-f4aa42813cb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:25.370748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2a194-c9f1-4f6a-811b-f4aa42813cb5 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:25.381401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1372/2772] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:32:32.142925 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a3868-f6f4-445c-a178-fd2b86e74833 service cinder] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:32.211456 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a3868-f6f4-445c-a178-fd2b86e74833 service cinder] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:32.213898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763193f6-d325-40f6-8e00-f543b158ce12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:32.214533 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763193f6-d325-40f6-8e00-f543b158ce12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:32.214870 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763193f6-d325-40f6-8e00-f543b158ce12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:32.215154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763193f6-d325-40f6-8e00-f543b158ce12 admin admin] PATH_INFO: `/v3/users/33ae07492df24916ab68e37194b086f9` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:32.216290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763193f6-d325-40f6-8e00-f543b158ce12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33ae07492df24916ab68e37194b086f9)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:32.218269 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763193f6-d325-40f6-8e00-f543b158ce12 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:32.422433 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-763193f6-d325-40f6-8e00-f543b158ce12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33ae07492df24916ab68e37194b086f9'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:32.482137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-763193f6-d325-40f6-8e00-f543b158ce12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33ae07492df24916ab68e37194b086f9'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:32.482784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-763193f6-d325-40f6-8e00-f543b158ce12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33ae07492df24916ab68e37194b086f9'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:32.490056 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-763193f6-d325-40f6-8e00-f543b158ce12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33ae07492df24916ab68e37194b086f9'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:32.502170 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-763193f6-d325-40f6-8e00-f543b158ce12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33ae07492df24916ab68e37194b086f9'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:32.507997 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1401/2773] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:32:32 2022] DELETE /identity/v3/users/33ae07492df24916ab68e37194b086f9 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:32.513059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef664e-083f-463f-acf5-df831ecb9518 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:32.517752 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef664e-083f-463f-acf5-df831ecb9518 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:32.517752 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef664e-083f-463f-acf5-df831ecb9518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:32.533313 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-c1ef664e-083f-463f-acf5-df831ecb9518 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:32.534514 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c1ef664e-083f-463f-acf5-df831ecb9518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:32.593294 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1373/2774] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:32.603001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763193f6-d325-40f6-8e00-f543b158ce12 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:32.604381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-763193f6-d325-40f6-8e00-f543b158ce12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:32.630618 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:32.661279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763193f6-d325-40f6-8e00-f543b158ce12 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:32.664959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a8aa3-9b34-4369-a539-5179d6329da2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:32.665663 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a8aa3-9b34-4369-a539-5179d6329da2 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:32.665952 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a8aa3-9b34-4369-a539-5179d6329da2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:32.666218 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a8aa3-9b34-4369-a539-5179d6329da2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:32.675310 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a8aa3-9b34-4369-a539-5179d6329da2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:32.676782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a8aa3-9b34-4369-a539-5179d6329da2 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:32.686944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1402/2775] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:32:33.084512 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef664e-083f-463f-acf5-df831ecb9518 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:33.086384 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c1ef664e-083f-463f-acf5-df831ecb9518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:33.107719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef664e-083f-463f-acf5-df831ecb9518 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:33.123761 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c3f83-9327-49bb-814d-dcb19925c729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:33.124561 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c3f83-9327-49bb-814d-dcb19925c729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:33.124860 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c3f83-9327-49bb-814d-dcb19925c729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:33.125137 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c3f83-9327-49bb-814d-dcb19925c729 admin admin] PATH_INFO: `/v3/projects/ae03acd916ee471a871dce2bccc399c7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:33.126385 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c3f83-9327-49bb-814d-dcb19925c729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae03acd916ee471a871dce2bccc399c7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:33.128489 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c3f83-9327-49bb-814d-dcb19925c729 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:33.156076 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:33.344108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6c0c3f83-9327-49bb-814d-dcb19925c729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae03acd916ee471a871dce2bccc399c7', 'request_id': 'req-6c0c3f83-9327-49bb-814d-dcb19925c729'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:33.349546 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6c0c3f83-9327-49bb-814d-dcb19925c729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae03acd916ee471a871dce2bccc399c7', 'request_id': 'req-6c0c3f83-9327-49bb-814d-dcb19925c729'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:33.356991 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6c0c3f83-9327-49bb-814d-dcb19925c729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae03acd916ee471a871dce2bccc399c7', 'request_id': 'req-6c0c3f83-9327-49bb-814d-dcb19925c729'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:33.432510 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6c0c3f83-9327-49bb-814d-dcb19925c729 admin admin] The token cache is being invalidate because project ae03acd916ee471a871dce2bccc399c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:32:33.433245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6c0c3f83-9327-49bb-814d-dcb19925c729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:33.433519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-6c0c3f83-9327-49bb-814d-dcb19925c729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:32:33.435018 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1374/2776] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:32:33 2022] DELETE /identity/v3/projects/ae03acd916ee471a871dce2bccc399c7 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:33.442679 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec2be6-306d-40ab-aeba-55a5548c49f0 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:33.443207 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec2be6-306d-40ab-aeba-55a5548c49f0 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:33.443324 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec2be6-306d-40ab-aeba-55a5548c49f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:33.466125 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-38ec2be6-306d-40ab-aeba-55a5548c49f0 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:33.467934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-38ec2be6-306d-40ab-aeba-55a5548c49f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:33.534380 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1403/2777] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:33.603063 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7701b90-6828-4e48-af26-b2a4c4ac826e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:33.604002 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7701b90-6828-4e48-af26-b2a4c4ac826e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:33.604285 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7701b90-6828-4e48-af26-b2a4c4ac826e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:33.631487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-a7701b90-6828-4e48-af26-b2a4c4ac826e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:33.632719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a7701b90-6828-4e48-af26-b2a4c4ac826e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:33.640831 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1375/2778] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:33.645225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec2be6-306d-40ab-aeba-55a5548c49f0 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:33.656194 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec2be6-306d-40ab-aeba-55a5548c49f0 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:33.661197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be50a2-17fa-4b27-86d6-f39b29c19d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:33.661812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be50a2-17fa-4b27-86d6-f39b29c19d31 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:33.662096 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be50a2-17fa-4b27-86d6-f39b29c19d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:33.662376 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be50a2-17fa-4b27-86d6-f39b29c19d31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:33.663047 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be50a2-17fa-4b27-86d6-f39b29c19d31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:32:33.663353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be50a2-17fa-4b27-86d6-f39b29c19d31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:33.666847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be50a2-17fa-4b27-86d6-f39b29c19d31 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:33.679836 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1404/2779] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:32:33.685673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7701b90-6828-4e48-af26-b2a4c4ac826e None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:33.699481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7701b90-6828-4e48-af26-b2a4c4ac826e None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:33.705468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745ce39-7ebd-464b-ae20-13071997c4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:33.706071 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745ce39-7ebd-464b-ae20-13071997c4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:33.706351 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745ce39-7ebd-464b-ae20-13071997c4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:33.706623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745ce39-7ebd-464b-ae20-13071997c4f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:33.707258 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745ce39-7ebd-464b-ae20-13071997c4f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:33.712443 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745ce39-7ebd-464b-ae20-13071997c4f5 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:33.817040 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1376/2780] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:32:33 2022] POST /identity/v3/projects => generated 361 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:32:33.821757 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be50a2-17fa-4b27-86d6-f39b29c19d31 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:33.831917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be50a2-17fa-4b27-86d6-f39b29c19d31 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:33.906828 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0b35d-cba5-4f01-8491-65fce890433b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:33.910845 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0b35d-cba5-4f01-8491-65fce890433b admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:33.910845 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0b35d-cba5-4f01-8491-65fce890433b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:33.910845 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0b35d-cba5-4f01-8491-65fce890433b admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:33.910845 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0b35d-cba5-4f01-8491-65fce890433b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:33.910845 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0b35d-cba5-4f01-8491-65fce890433b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:33.993045 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1405/2781] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:32:33 2022] POST /identity/v3/users => generated 281 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:32:33.997781 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745ce39-7ebd-464b-ae20-13071997c4f5 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:34.007981 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745ce39-7ebd-464b-ae20-13071997c4f5 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:34.012901 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba925835-3fb0-4207-a8fd-1c6eaa22b43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:34.013513 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba925835-3fb0-4207-a8fd-1c6eaa22b43f admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:34.013774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba925835-3fb0-4207-a8fd-1c6eaa22b43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:34.014048 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba925835-3fb0-4207-a8fd-1c6eaa22b43f admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:34.014745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba925835-3fb0-4207-a8fd-1c6eaa22b43f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:34.016402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba925835-3fb0-4207-a8fd-1c6eaa22b43f admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:34.026990 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1377/2782] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:32:34.031153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0b35d-cba5-4f01-8491-65fce890433b admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:34.041984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0b35d-cba5-4f01-8491-65fce890433b admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:34.046599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcdc4d9-b353-45c6-9779-df6b1991755d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:34.047187 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdc4d9-b353-45c6-9779-df6b1991755d admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:34.047470 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdc4d9-b353-45c6-9779-df6b1991755d admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:34.047746 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcdc4d9-b353-45c6-9779-df6b1991755d admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:34.048386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcdc4d9-b353-45c6-9779-df6b1991755d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:34.050012 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcdc4d9-b353-45c6-9779-df6b1991755d admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:34.061167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1406/2783] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:32:34.065239 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba925835-3fb0-4207-a8fd-1c6eaa22b43f admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:34.075044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba925835-3fb0-4207-a8fd-1c6eaa22b43f admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:34.079570 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953d147-4a23-4ab3-9667-3cab41bdf944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:34.080633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953d147-4a23-4ab3-9667-3cab41bdf944 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:34.080944 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953d147-4a23-4ab3-9667-3cab41bdf944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:34.083773 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953d147-4a23-4ab3-9667-3cab41bdf944 admin admin] PATH_INFO: `/v3/projects/a194328bf98443bc9e6d2b337a8233d7/users/563ee9b635854f3fa70eae9564d14554/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:34.096616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953d147-4a23-4ab3-9667-3cab41bdf944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a194328bf98443bc9e6d2b337a8233d7, user_id=563ee9b635854f3fa70eae9564d14554, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:34.099328 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953d147-4a23-4ab3-9667-3cab41bdf944 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:34.192030 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1378/2784] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:32:34 2022] PUT /identity/v3/projects/a194328bf98443bc9e6d2b337a8233d7/users/563ee9b635854f3fa70eae9564d14554/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:32:34.202350 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb6bc6-5787-4b76-b434-7a922eda7640 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:34.202646 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb6bc6-5787-4b76-b434-7a922eda7640 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:34.202931 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb6bc6-5787-4b76-b434-7a922eda7640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:34.231720 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-c2cb6bc6-5787-4b76-b434-7a922eda7640 None None] MFA Rules not processed for user `563ee9b635854f3fa70eae9564d14554`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:34.233045 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-c2cb6bc6-5787-4b76-b434-7a922eda7640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:34.251482 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:34.299292 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1407/2785] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:34.311481 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953d147-4a23-4ab3-9667-3cab41bdf944 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:32:34.313161 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8953d147-4a23-4ab3-9667-3cab41bdf944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:34.370166 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953d147-4a23-4ab3-9667-3cab41bdf944 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:32:34.374091 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c7a57-6f19-4784-b5a0-682e079f80cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:32:34.374691 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c7a57-6f19-4784-b5a0-682e079f80cc service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:34.374978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c7a57-6f19-4784-b5a0-682e079f80cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:34.375248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c7a57-6f19-4784-b5a0-682e079f80cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:34.383865 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c7a57-6f19-4784-b5a0-682e079f80cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:32:34.385322 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c7a57-6f19-4784-b5a0-682e079f80cc service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:32:34.393932 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1379/2786] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:32:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:32:37.395144 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e22766-1385-4579-aa49-9e96a705730d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:32:37.395605 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e22766-1385-4579-aa49-9e96a705730d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:32:37.395712 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e22766-1385-4579-aa49-9e96a705730d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:32:37.415747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-b0e22766-1385-4579-aa49-9e96a705730d None None] MFA Rules not processed for user `a90e3591b4144fdfb8b5ed5391aec7c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:32:37.417001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b0e22766-1385-4579-aa49-9e96a705730d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:32:37.474615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1408/2787] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:32:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:32:44.390032 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:32:47.467671 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=43 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:02.908858 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c7a57-6f19-4784-b5a0-682e079f80cc service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:02.912178 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-5e4c7a57-6f19-4784-b5a0-682e079f80cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:02.980253 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c7a57-6f19-4784-b5a0-682e079f80cc service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:02.984749 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b3d0f-fae8-48a8-92c5-16362e448827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:02.985366 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b3d0f-fae8-48a8-92c5-16362e448827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:02.985648 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b3d0f-fae8-48a8-92c5-16362e448827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:02.985919 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b3d0f-fae8-48a8-92c5-16362e448827 admin admin] PATH_INFO: `/v3/users/910098074b83416fbf14664ef99d9227` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:02.987044 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b3d0f-fae8-48a8-92c5-16362e448827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=910098074b83416fbf14664ef99d9227)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:02.988989 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b3d0f-fae8-48a8-92c5-16362e448827 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:03.281252 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2b8b3d0f-fae8-48a8-92c5-16362e448827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '910098074b83416fbf14664ef99d9227'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:03.281917 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2b8b3d0f-fae8-48a8-92c5-16362e448827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '910098074b83416fbf14664ef99d9227'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:03.368142 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2b8b3d0f-fae8-48a8-92c5-16362e448827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '910098074b83416fbf14664ef99d9227'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:03.381279 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2b8b3d0f-fae8-48a8-92c5-16362e448827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '910098074b83416fbf14664ef99d9227'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:03.388395 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-2b8b3d0f-fae8-48a8-92c5-16362e448827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '910098074b83416fbf14664ef99d9227'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:03.394304 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1380/2788] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:33:02 2022] DELETE /identity/v3/users/910098074b83416fbf14664ef99d9227 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:03.638603 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e22766-1385-4579-aa49-9e96a705730d None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:03.652969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e22766-1385-4579-aa49-9e96a705730d None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:03.710163 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dd500-4448-486f-8278-431a00151358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:03.710812 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dd500-4448-486f-8278-431a00151358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:03.711095 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dd500-4448-486f-8278-431a00151358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:03.711623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dd500-4448-486f-8278-431a00151358 admin admin] PATH_INFO: `/v3/projects/5dded162e47d494d9683568544d446bb` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:03.713288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7dd500-4448-486f-8278-431a00151358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dded162e47d494d9683568544d446bb)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:03.715172 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7dd500-4448-486f-8278-431a00151358 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:03.849448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-9c7dd500-4448-486f-8278-431a00151358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dded162e47d494d9683568544d446bb', 'request_id': 'req-9c7dd500-4448-486f-8278-431a00151358'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:03.855721 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-9c7dd500-4448-486f-8278-431a00151358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dded162e47d494d9683568544d446bb', 'request_id': 'req-9c7dd500-4448-486f-8278-431a00151358'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:03.861992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-9c7dd500-4448-486f-8278-431a00151358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dded162e47d494d9683568544d446bb', 'request_id': 'req-9c7dd500-4448-486f-8278-431a00151358'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:03.956880 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-9c7dd500-4448-486f-8278-431a00151358 admin admin] The token cache is being invalidate because project 5dded162e47d494d9683568544d446bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:33:03.957177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-9c7dd500-4448-486f-8278-431a00151358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:03.957800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-9c7dd500-4448-486f-8278-431a00151358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:03.959276 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1409/2789] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:33:03 2022] DELETE /identity/v3/projects/5dded162e47d494d9683568544d446bb => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:03.963823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b3d0f-fae8-48a8-92c5-16362e448827 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:03.965230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-2b8b3d0f-fae8-48a8-92c5-16362e448827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:03.978945 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b3d0f-fae8-48a8-92c5-16362e448827 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:04.034210 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f955bc-b7dd-4485-9161-3507399c3b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:04.034828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f955bc-b7dd-4485-9161-3507399c3b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:04.035112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f955bc-b7dd-4485-9161-3507399c3b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:04.035389 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f955bc-b7dd-4485-9161-3507399c3b78 admin admin] PATH_INFO: `/v3/users/311ae2a7af7b464e898432619ed3f449` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:04.036516 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f955bc-b7dd-4485-9161-3507399c3b78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=311ae2a7af7b464e898432619ed3f449)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:04.038459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f955bc-b7dd-4485-9161-3507399c3b78 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:04.060777 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:04.268533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-80f955bc-b7dd-4485-9161-3507399c3b78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '311ae2a7af7b464e898432619ed3f449'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:04.269247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-80f955bc-b7dd-4485-9161-3507399c3b78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '311ae2a7af7b464e898432619ed3f449'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:04.339230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-80f955bc-b7dd-4485-9161-3507399c3b78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '311ae2a7af7b464e898432619ed3f449'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:04.353268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-80f955bc-b7dd-4485-9161-3507399c3b78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '311ae2a7af7b464e898432619ed3f449'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:04.360729 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-80f955bc-b7dd-4485-9161-3507399c3b78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '311ae2a7af7b464e898432619ed3f449'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:04.367037 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1381/2790] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:33:03 2022] DELETE /identity/v3/users/311ae2a7af7b464e898432619ed3f449 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:04.621538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dd500-4448-486f-8278-431a00151358 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:04.623028 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-9c7dd500-4448-486f-8278-431a00151358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:04.639236 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dd500-4448-486f-8278-431a00151358 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:04.696206 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a72ddb-8417-46f1-97d5-786e50ea9b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:04.697359 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a72ddb-8417-46f1-97d5-786e50ea9b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:04.697637 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a72ddb-8417-46f1-97d5-786e50ea9b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:04.697906 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a72ddb-8417-46f1-97d5-786e50ea9b4c admin admin] PATH_INFO: `/v3/projects/8da1de6892394064a0f3b8cf171b9bbc` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:04.699007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a72ddb-8417-46f1-97d5-786e50ea9b4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8da1de6892394064a0f3b8cf171b9bbc)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:04.700944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a72ddb-8417-46f1-97d5-786e50ea9b4c admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:04.814390 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:04.826477 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-88a72ddb-8417-46f1-97d5-786e50ea9b4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8da1de6892394064a0f3b8cf171b9bbc', 'request_id': 'req-88a72ddb-8417-46f1-97d5-786e50ea9b4c'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:04.831967 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-88a72ddb-8417-46f1-97d5-786e50ea9b4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8da1de6892394064a0f3b8cf171b9bbc', 'request_id': 'req-88a72ddb-8417-46f1-97d5-786e50ea9b4c'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:04.839162 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-88a72ddb-8417-46f1-97d5-786e50ea9b4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8da1de6892394064a0f3b8cf171b9bbc', 'request_id': 'req-88a72ddb-8417-46f1-97d5-786e50ea9b4c'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:04.950650 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-88a72ddb-8417-46f1-97d5-786e50ea9b4c admin admin] The token cache is being invalidate because project 8da1de6892394064a0f3b8cf171b9bbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:33:04.950937 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-88a72ddb-8417-46f1-97d5-786e50ea9b4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:04.951593 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-88a72ddb-8417-46f1-97d5-786e50ea9b4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:04.953102 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1410/2791] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:33:04 2022] DELETE /identity/v3/projects/8da1de6892394064a0f3b8cf171b9bbc => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:04.957987 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c43780-a2dc-4afb-aa3e-0251b4e2561e None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:04.958286 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c43780-a2dc-4afb-aa3e-0251b4e2561e None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:04.958560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c43780-a2dc-4afb-aa3e-0251b4e2561e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:04.978454 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-a4c43780-a2dc-4afb-aa3e-0251b4e2561e None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:04.979708 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a4c43780-a2dc-4afb-aa3e-0251b4e2561e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:05.038360 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1382/2792] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:05.047494 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97664352-3a3f-4b45-8a7b-a05a96743d5d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:05.047782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97664352-3a3f-4b45-8a7b-a05a96743d5d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:05.048059 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97664352-3a3f-4b45-8a7b-a05a96743d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:05.067503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-97664352-3a3f-4b45-8a7b-a05a96743d5d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:05.068723 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-97664352-3a3f-4b45-8a7b-a05a96743d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:05.078916 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1411/2793] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:05.083023 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c43780-a2dc-4afb-aa3e-0251b4e2561e None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:05.093150 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c43780-a2dc-4afb-aa3e-0251b4e2561e None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:05.097703 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae1a18-4430-4d06-a90c-285a65987a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:05.098302 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae1a18-4430-4d06-a90c-285a65987a99 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:05.098607 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae1a18-4430-4d06-a90c-285a65987a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:05.098879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae1a18-4430-4d06-a90c-285a65987a99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:05.099519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae1a18-4430-4d06-a90c-285a65987a99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:33:05.099821 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae1a18-4430-4d06-a90c-285a65987a99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:05.101455 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae1a18-4430-4d06-a90c-285a65987a99 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:05.112972 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1383/2794] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:33:05.117044 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97664352-3a3f-4b45-8a7b-a05a96743d5d None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:05.126623 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97664352-3a3f-4b45-8a7b-a05a96743d5d None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:05.131149 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a599213-ddc2-4c6e-bd89-7b9c59411ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:05.131731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a599213-ddc2-4c6e-bd89-7b9c59411ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:05.132011 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a599213-ddc2-4c6e-bd89-7b9c59411ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:05.132279 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a599213-ddc2-4c6e-bd89-7b9c59411ad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:05.132914 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a599213-ddc2-4c6e-bd89-7b9c59411ad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:05.135465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a599213-ddc2-4c6e-bd89-7b9c59411ad9 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:05.238239 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1412/2795] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:33:05 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:33:05.242227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae1a18-4430-4d06-a90c-285a65987a99 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:05.252621 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae1a18-4430-4d06-a90c-285a65987a99 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:05.308457 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b009e-ed69-494a-b56a-408d19592239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:05.309084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b009e-ed69-494a-b56a-408d19592239 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:05.309363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b009e-ed69-494a-b56a-408d19592239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:05.309636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b009e-ed69-494a-b56a-408d19592239 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:05.310259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b009e-ed69-494a-b56a-408d19592239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:05.312140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b009e-ed69-494a-b56a-408d19592239 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:05.414236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1384/2796] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:33:05 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:33:05.418902 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a599213-ddc2-4c6e-bd89-7b9c59411ad9 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:05.430753 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a599213-ddc2-4c6e-bd89-7b9c59411ad9 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:05.436298 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba7f2a-68fa-42f2-a072-fe1888f16ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:05.452148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba7f2a-68fa-42f2-a072-fe1888f16ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:05.452148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba7f2a-68fa-42f2-a072-fe1888f16ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:05.452148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba7f2a-68fa-42f2-a072-fe1888f16ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:05.452148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba7f2a-68fa-42f2-a072-fe1888f16ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:05.452148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba7f2a-68fa-42f2-a072-fe1888f16ac4 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:05.452148 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1413/2797] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:33:05.454803 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b009e-ed69-494a-b56a-408d19592239 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:05.465497 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b009e-ed69-494a-b56a-408d19592239 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:05.471059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbbda8-2e78-4a1d-a896-45dad68bc17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:05.471666 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbbda8-2e78-4a1d-a896-45dad68bc17a admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:05.471950 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbbda8-2e78-4a1d-a896-45dad68bc17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:05.472228 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbbda8-2e78-4a1d-a896-45dad68bc17a admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:05.472910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbbda8-2e78-4a1d-a896-45dad68bc17a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:05.474544 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbbda8-2e78-4a1d-a896-45dad68bc17a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:05.487491 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1385/2798] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:33:05.491958 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba7f2a-68fa-42f2-a072-fe1888f16ac4 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:05.502604 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba7f2a-68fa-42f2-a072-fe1888f16ac4 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:05.507483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7295a-029d-443c-8395-2ba96cf467d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:05.508131 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7295a-029d-443c-8395-2ba96cf467d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:05.508414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7295a-029d-443c-8395-2ba96cf467d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:05.508721 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7295a-029d-443c-8395-2ba96cf467d6 admin admin] PATH_INFO: `/v3/projects/063685edf9af41989d027e6bd635eedd/users/5e059a2eddd047be8b77bc298c5d82c5/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:05.524271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7295a-029d-443c-8395-2ba96cf467d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=063685edf9af41989d027e6bd635eedd, user_id=5e059a2eddd047be8b77bc298c5d82c5, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:05.527056 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7295a-029d-443c-8395-2ba96cf467d6 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:05.609632 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1414/2799] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:33:05 2022] PUT /identity/v3/projects/063685edf9af41989d027e6bd635eedd/users/5e059a2eddd047be8b77bc298c5d82c5/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:05.620673 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bac8b-2ccc-4b05-aa71-89d3c784eae4 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:05.621474 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bac8b-2ccc-4b05-aa71-89d3c784eae4 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:05.621678 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bac8b-2ccc-4b05-aa71-89d3c784eae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:05.630864 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:05.649334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-a54bac8b-2ccc-4b05-aa71-89d3c784eae4 None None] MFA Rules not processed for user `5e059a2eddd047be8b77bc298c5d82c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:05.651225 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a54bac8b-2ccc-4b05-aa71-89d3c784eae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:05.714320 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1386/2800] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:33:05.725566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7295a-029d-443c-8395-2ba96cf467d6 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:05.727246 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e2b7295a-029d-443c-8395-2ba96cf467d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:05.784448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7295a-029d-443c-8395-2ba96cf467d6 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:05.788135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334a858-889d-4782-bfbf-2dd69128b60c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:05.788779 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334a858-889d-4782-bfbf-2dd69128b60c service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:05.789068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334a858-889d-4782-bfbf-2dd69128b60c service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:05.789344 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334a858-889d-4782-bfbf-2dd69128b60c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:05.799009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334a858-889d-4782-bfbf-2dd69128b60c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:05.800755 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334a858-889d-4782-bfbf-2dd69128b60c service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:05.810998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1415/2801] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:33:08.939301 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bac8b-2ccc-4b05-aa71-89d3c784eae4 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:08.942146 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a54bac8b-2ccc-4b05-aa71-89d3c784eae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:09.016748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bac8b-2ccc-4b05-aa71-89d3c784eae4 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:09.022487 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb1df6-d1ba-41af-be19-98e7f1335c50 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ee8ba21bc184feeaafacab3e69e87bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:09.023611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb1df6-d1ba-41af-be19-98e7f1335c50 service placement] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:09.024184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb1df6-d1ba-41af-be19-98e7f1335c50 service placement] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:09.024618 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb1df6-d1ba-41af-be19-98e7f1335c50 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:09.027410 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9edb1df6-d1ba-41af-be19-98e7f1335c50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:09.045190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb1df6-d1ba-41af-be19-98e7f1335c50 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:09.045190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb1df6-d1ba-41af-be19-98e7f1335c50 service placement] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:09.057849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1387/2802] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:33:09.377475 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334a858-889d-4782-bfbf-2dd69128b60c service cinder] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:09.379190 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b334a858-889d-4782-bfbf-2dd69128b60c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:09.443291 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334a858-889d-4782-bfbf-2dd69128b60c service cinder] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:09.449132 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca6927-197a-4e79-aae4-accacc588015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:09.449762 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca6927-197a-4e79-aae4-accacc588015 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:09.450029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca6927-197a-4e79-aae4-accacc588015 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:09.450304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca6927-197a-4e79-aae4-accacc588015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:09.452146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-75ca6927-197a-4e79-aae4-accacc588015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:09.465799 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:09.516969 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca6927-197a-4e79-aae4-accacc588015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:09.518442 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca6927-197a-4e79-aae4-accacc588015 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:09.528763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1416/2803] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:33:10.269688 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb1df6-d1ba-41af-be19-98e7f1335c50 service placement] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:10.347557 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb1df6-d1ba-41af-be19-98e7f1335c50 service placement] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:10.352360 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcab3a-cfc8-49ec-ada7-57202148afce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:10.353830 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcab3a-cfc8-49ec-ada7-57202148afce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:10.357878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcab3a-cfc8-49ec-ada7-57202148afce admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:10.357878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcab3a-cfc8-49ec-ada7-57202148afce admin admin] PATH_INFO: `/v3/users/5e059a2eddd047be8b77bc298c5d82c5` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:10.357878 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bcab3a-cfc8-49ec-ada7-57202148afce admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e059a2eddd047be8b77bc298c5d82c5)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:10.358334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bcab3a-cfc8-49ec-ada7-57202148afce admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:10.466495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:10.554267 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-72bcab3a-cfc8-49ec-ada7-57202148afce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e059a2eddd047be8b77bc298c5d82c5'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:10.554957 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-72bcab3a-cfc8-49ec-ada7-57202148afce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e059a2eddd047be8b77bc298c5d82c5'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:10.654657 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-72bcab3a-cfc8-49ec-ada7-57202148afce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e059a2eddd047be8b77bc298c5d82c5'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:10.668534 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-72bcab3a-cfc8-49ec-ada7-57202148afce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e059a2eddd047be8b77bc298c5d82c5'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:10.675521 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-72bcab3a-cfc8-49ec-ada7-57202148afce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e059a2eddd047be8b77bc298c5d82c5'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:10.682524 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1388/2804] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:33:10 2022] DELETE /identity/v3/users/5e059a2eddd047be8b77bc298c5d82c5 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:10.686990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4de606-4ace-4277-8b7f-71da55a54518 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:10.687497 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4de606-4ace-4277-8b7f-71da55a54518 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:10.687677 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4de606-4ace-4277-8b7f-71da55a54518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:10.707826 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-fb4de606-4ace-4277-8b7f-71da55a54518 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:10.709069 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-fb4de606-4ace-4277-8b7f-71da55a54518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:10.792768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1417/2805] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:10.805334 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcab3a-cfc8-49ec-ada7-57202148afce admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:10.807577 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-72bcab3a-cfc8-49ec-ada7-57202148afce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:10.869613 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcab3a-cfc8-49ec-ada7-57202148afce admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:10.873516 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd0469-8a28-490b-819a-ecf9f45565bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:10.874119 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd0469-8a28-490b-819a-ecf9f45565bd service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:10.874397 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd0469-8a28-490b-819a-ecf9f45565bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:10.874669 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd0469-8a28-490b-819a-ecf9f45565bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:10.883859 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd0469-8a28-490b-819a-ecf9f45565bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:10.885465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd0469-8a28-490b-819a-ecf9f45565bd service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:10.894748 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1389/2806] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:33:11.495357 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4de606-4ace-4277-8b7f-71da55a54518 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:11.496935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-fb4de606-4ace-4277-8b7f-71da55a54518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:11.509607 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4de606-4ace-4277-8b7f-71da55a54518 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:11.516444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681917b-003b-4604-9a12-0af47be899e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:11.517125 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681917b-003b-4604-9a12-0af47be899e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:11.517454 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681917b-003b-4604-9a12-0af47be899e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:11.522622 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681917b-003b-4604-9a12-0af47be899e2 admin admin] PATH_INFO: `/v3/projects/063685edf9af41989d027e6bd635eedd` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:11.529414 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681917b-003b-4604-9a12-0af47be899e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=063685edf9af41989d027e6bd635eedd)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:11.531347 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681917b-003b-4604-9a12-0af47be899e2 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:11.681952 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d681917b-003b-4604-9a12-0af47be899e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '063685edf9af41989d027e6bd635eedd', 'request_id': 'req-d681917b-003b-4604-9a12-0af47be899e2'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:11.689413 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d681917b-003b-4604-9a12-0af47be899e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '063685edf9af41989d027e6bd635eedd', 'request_id': 'req-d681917b-003b-4604-9a12-0af47be899e2'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:11.697509 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d681917b-003b-4604-9a12-0af47be899e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '063685edf9af41989d027e6bd635eedd', 'request_id': 'req-d681917b-003b-4604-9a12-0af47be899e2'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:11.778029 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd0469-8a28-490b-819a-ecf9f45565bd service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:11.779482 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c3dd0469-8a28-490b-819a-ecf9f45565bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:11.837995 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d681917b-003b-4604-9a12-0af47be899e2 admin admin] The token cache is being invalidate because project 063685edf9af41989d027e6bd635eedd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:33:11.838483 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d681917b-003b-4604-9a12-0af47be899e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:11.840959 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-d681917b-003b-4604-9a12-0af47be899e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:11.843823 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1418/2807] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:33:11 2022] DELETE /identity/v3/projects/063685edf9af41989d027e6bd635eedd => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:11.849541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf463268-4c56-4147-9027-228e12e65c25 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:11.849934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf463268-4c56-4147-9027-228e12e65c25 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:11.850289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf463268-4c56-4147-9027-228e12e65c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:11.867530 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd0469-8a28-490b-819a-ecf9f45565bd service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:11.876769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-cf463268-4c56-4147-9027-228e12e65c25 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:11.878137 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-cf463268-4c56-4147-9027-228e12e65c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:11.928856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:11.933946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:11.959143 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80142d0-a0fe-4ae5-b797-ac9ee3ca2179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:11.959709 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80142d0-a0fe-4ae5-b797-ac9ee3ca2179 service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:11.959986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80142d0-a0fe-4ae5-b797-ac9ee3ca2179 service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:11.960518 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80142d0-a0fe-4ae5-b797-ac9ee3ca2179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:11.962742 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-a80142d0-a0fe-4ae5-b797-ac9ee3ca2179 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:11.970360 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1419/2808] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:33:11.972425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d0712-5203-4307-8428-07273a981c61 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:11.972786 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d0712-5203-4307-8428-07273a981c61 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:11.973017 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d0712-5203-4307-8428-07273a981c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:11.991840 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80142d0-a0fe-4ae5-b797-ac9ee3ca2179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:11.993419 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80142d0-a0fe-4ae5-b797-ac9ee3ca2179 service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:11.996669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-db7d0712-5203-4307-8428-07273a981c61 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:11.997531 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-db7d0712-5203-4307-8428-07273a981c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:12.009404 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1420/2809] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:12.013975 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d0712-5203-4307-8428-07273a981c61 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:12.017793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1390/2810] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:33:12.025926 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d0712-5203-4307-8428-07273a981c61 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:12.031620 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd01ce8-0aa6-4194-9969-7f0c85b22626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:12.032444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd01ce8-0aa6-4194-9969-7f0c85b22626 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:12.032899 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd01ce8-0aa6-4194-9969-7f0c85b22626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:12.033599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd01ce8-0aa6-4194-9969-7f0c85b22626 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:12.034025 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd01ce8-0aa6-4194-9969-7f0c85b22626 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:33:12.040029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd01ce8-0aa6-4194-9969-7f0c85b22626 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:12.040029 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd01ce8-0aa6-4194-9969-7f0c85b22626 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:12.054597 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1421/2811] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:33:12.056970 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80142d0-a0fe-4ae5-b797-ac9ee3ca2179 service nova] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:12.073380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80142d0-a0fe-4ae5-b797-ac9ee3ca2179 service nova] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:12.078317 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31359c56-3592-4dd9-a3db-5e7ca3d56190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:12.078914 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31359c56-3592-4dd9-a3db-5e7ca3d56190 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:12.079190 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31359c56-3592-4dd9-a3db-5e7ca3d56190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:12.079459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31359c56-3592-4dd9-a3db-5e7ca3d56190 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:12.080083 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31359c56-3592-4dd9-a3db-5e7ca3d56190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:12.082287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31359c56-3592-4dd9-a3db-5e7ca3d56190 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:12.202636 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1391/2812] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:33:12 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:33:12.207102 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd01ce8-0aa6-4194-9969-7f0c85b22626 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:12.219581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd01ce8-0aa6-4194-9969-7f0c85b22626 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:12.315907 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8026ad-9b82-424b-8a65-bb8186bc1a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:12.316524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8026ad-9b82-424b-8a65-bb8186bc1a69 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:12.317328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8026ad-9b82-424b-8a65-bb8186bc1a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:12.317328 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8026ad-9b82-424b-8a65-bb8186bc1a69 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:12.317990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8026ad-9b82-424b-8a65-bb8186bc1a69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:12.319912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8026ad-9b82-424b-8a65-bb8186bc1a69 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:12.432228 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1422/2813] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:33:12 2022] POST /identity/v3/users => generated 287 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:33:12.436248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31359c56-3592-4dd9-a3db-5e7ca3d56190 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:12.450476 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31359c56-3592-4dd9-a3db-5e7ca3d56190 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:12.457670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc269c3-a5a0-4753-b938-b76937f58c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:12.457670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc269c3-a5a0-4753-b938-b76937f58c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:12.458238 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc269c3-a5a0-4753-b938-b76937f58c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:12.458676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc269c3-a5a0-4753-b938-b76937f58c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:12.459495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc269c3-a5a0-4753-b938-b76937f58c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:12.462512 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc269c3-a5a0-4753-b938-b76937f58c7d admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:12.476259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1392/2814] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:33:12.479773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8026ad-9b82-424b-8a65-bb8186bc1a69 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:12.490228 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8026ad-9b82-424b-8a65-bb8186bc1a69 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:12.496184 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67846c-2fa4-4236-800d-934427314f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:12.497001 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67846c-2fa4-4236-800d-934427314f36 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:12.497377 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67846c-2fa4-4236-800d-934427314f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:12.497751 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67846c-2fa4-4236-800d-934427314f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:12.498535 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67846c-2fa4-4236-800d-934427314f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:12.500238 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67846c-2fa4-4236-800d-934427314f36 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:12.517030 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1423/2815] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:33:12.517908 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc269c3-a5a0-4753-b938-b76937f58c7d admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:12.529525 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc269c3-a5a0-4753-b938-b76937f58c7d admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:12.536547 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59918da-4bdd-41d4-a8e0-6539a224efcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:12.537261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59918da-4bdd-41d4-a8e0-6539a224efcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:12.537535 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59918da-4bdd-41d4-a8e0-6539a224efcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:12.537823 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59918da-4bdd-41d4-a8e0-6539a224efcc admin admin] PATH_INFO: `/v3/projects/938bfb2e36e34b4283988e8b6bd9ee92/users/39ef1e3c0af4441c873b0b2d959f7b1b/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:12.557540 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59918da-4bdd-41d4-a8e0-6539a224efcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=938bfb2e36e34b4283988e8b6bd9ee92, user_id=39ef1e3c0af4441c873b0b2d959f7b1b, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:12.560774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59918da-4bdd-41d4-a8e0-6539a224efcc admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:12.655278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67846c-2fa4-4236-800d-934427314f36 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:12.660677 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-ec67846c-2fa4-4236-800d-934427314f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:12.676774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1393/2816] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:33:12 2022] PUT /identity/v3/projects/938bfb2e36e34b4283988e8b6bd9ee92/users/39ef1e3c0af4441c873b0b2d959f7b1b/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:33:12.688979 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bd774-5cdd-4266-9c83-74202e6b24fb None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:12.691201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bd774-5cdd-4266-9c83-74202e6b24fb None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:12.691201 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bd774-5cdd-4266-9c83-74202e6b24fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:12.694984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67846c-2fa4-4236-800d-934427314f36 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:12.727277 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-015bd774-5cdd-4266-9c83-74202e6b24fb None None] MFA Rules not processed for user `39ef1e3c0af4441c873b0b2d959f7b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:12.729719 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-015bd774-5cdd-4266-9c83-74202e6b24fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:12.775728 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f54e8-4d75-4f66-bd76-2b24d69f81fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:12.776346 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f54e8-4d75-4f66-bd76-2b24d69f81fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:12.776839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f54e8-4d75-4f66-bd76-2b24d69f81fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:12.777142 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f54e8-4d75-4f66-bd76-2b24d69f81fb admin admin] PATH_INFO: `/v3/users/15271a6b8ca44359be06059f365d5390` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:12.778864 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f54e8-4d75-4f66-bd76-2b24d69f81fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=15271a6b8ca44359be06059f365d5390)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:12.781225 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f54e8-4d75-4f66-bd76-2b24d69f81fb admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:12.829982 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1394/2817] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:33:12.845477 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015bd774-5cdd-4266-9c83-74202e6b24fb None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:12.846935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-015bd774-5cdd-4266-9c83-74202e6b24fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:12.905013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015bd774-5cdd-4266-9c83-74202e6b24fb None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:12.909134 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ecf296-b4bd-41d8-bf4e-fc17106446d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:12.910070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecf296-b4bd-41d8-bf4e-fc17106446d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:12.914465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecf296-b4bd-41d8-bf4e-fc17106446d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:12.914465 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecf296-b4bd-41d8-bf4e-fc17106446d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:12.921310 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ecf296-b4bd-41d8-bf4e-fc17106446d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:12.922744 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ecf296-b4bd-41d8-bf4e-fc17106446d2 service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:12.932631 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1395/2818] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:33:13.023841 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:13.041578 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-a26f54e8-4d75-4f66-bd76-2b24d69f81fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15271a6b8ca44359be06059f365d5390'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:13.105528 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-a26f54e8-4d75-4f66-bd76-2b24d69f81fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15271a6b8ca44359be06059f365d5390'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:13.106185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-a26f54e8-4d75-4f66-bd76-2b24d69f81fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15271a6b8ca44359be06059f365d5390'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:13.113524 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-a26f54e8-4d75-4f66-bd76-2b24d69f81fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15271a6b8ca44359be06059f365d5390'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:13.125784 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-a26f54e8-4d75-4f66-bd76-2b24d69f81fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15271a6b8ca44359be06059f365d5390'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:13.131650 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1424/2819] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:33:12 2022] DELETE /identity/v3/users/15271a6b8ca44359be06059f365d5390 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:13.136370 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e05353-b0ca-4fa9-ad3c-1dce73b4f1a7 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:13.136700 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e05353-b0ca-4fa9-ad3c-1dce73b4f1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:13.136982 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e05353-b0ca-4fa9-ad3c-1dce73b4f1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:13.145014 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:13.156399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-59e05353-b0ca-4fa9-ad3c-1dce73b4f1a7 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:13.157978 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-59e05353-b0ca-4fa9-ad3c-1dce73b4f1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:13.221450 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1396/2820] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:33:13.232213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f54e8-4d75-4f66-bd76-2b24d69f81fb admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:13.233657 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-a26f54e8-4d75-4f66-bd76-2b24d69f81fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:13.293898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f54e8-4d75-4f66-bd76-2b24d69f81fb admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:13.297927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda8516-c073-4222-b77f-36948f531cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:13.298568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda8516-c073-4222-b77f-36948f531cdb service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:13.298847 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda8516-c073-4222-b77f-36948f531cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:13.299120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda8516-c073-4222-b77f-36948f531cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:13.309542 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda8516-c073-4222-b77f-36948f531cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:13.311069 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda8516-c073-4222-b77f-36948f531cdb service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:13.322677 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1425/2821] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:33:13.782124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e05353-b0ca-4fa9-ad3c-1dce73b4f1a7 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:13.783623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-59e05353-b0ca-4fa9-ad3c-1dce73b4f1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:13.796199 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e05353-b0ca-4fa9-ad3c-1dce73b4f1a7 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:13.801920 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d5f95-3060-4108-8fee-0f70a4551258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:13.802468 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d5f95-3060-4108-8fee-0f70a4551258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:13.802935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d5f95-3060-4108-8fee-0f70a4551258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:13.803513 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d5f95-3060-4108-8fee-0f70a4551258 admin admin] PATH_INFO: `/v3/projects/59704ec2c2fd427b80ebb60a8a9f9b6b` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:13.805207 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d5f95-3060-4108-8fee-0f70a4551258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59704ec2c2fd427b80ebb60a8a9f9b6b)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:13.807893 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d5f95-3060-4108-8fee-0f70a4551258 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:13.978839 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-439d5f95-3060-4108-8fee-0f70a4551258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59704ec2c2fd427b80ebb60a8a9f9b6b', 'request_id': 'req-439d5f95-3060-4108-8fee-0f70a4551258'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:13.984273 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-439d5f95-3060-4108-8fee-0f70a4551258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59704ec2c2fd427b80ebb60a8a9f9b6b', 'request_id': 'req-439d5f95-3060-4108-8fee-0f70a4551258'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:13.990431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-439d5f95-3060-4108-8fee-0f70a4551258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59704ec2c2fd427b80ebb60a8a9f9b6b', 'request_id': 'req-439d5f95-3060-4108-8fee-0f70a4551258'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:14.145124 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-439d5f95-3060-4108-8fee-0f70a4551258 admin admin] The token cache is being invalidate because project 59704ec2c2fd427b80ebb60a8a9f9b6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:33:14.145364 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-439d5f95-3060-4108-8fee-0f70a4551258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:14.145984 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-439d5f95-3060-4108-8fee-0f70a4551258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:14.147500 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1397/2822] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:33:13 2022] DELETE /identity/v3/projects/59704ec2c2fd427b80ebb60a8a9f9b6b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:14.151628 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda8516-c073-4222-b77f-36948f531cdb service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:14.161254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda8516-c073-4222-b77f-36948f531cdb service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:14.217884 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1412083-bc77-401a-b1c1-20c79d85230a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:14.218591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1412083-bc77-401a-b1c1-20c79d85230a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:14.218872 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1412083-bc77-401a-b1c1-20c79d85230a admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:14.219146 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1412083-bc77-401a-b1c1-20c79d85230a admin admin] PATH_INFO: `/v3/users/b71e6e57f8f34e9598e0a8c2e2e7a228` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:14.220247 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1412083-bc77-401a-b1c1-20c79d85230a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b71e6e57f8f34e9598e0a8c2e2e7a228)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:14.222177 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1412083-bc77-401a-b1c1-20c79d85230a admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:14.461747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f1412083-bc77-401a-b1c1-20c79d85230a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b71e6e57f8f34e9598e0a8c2e2e7a228'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:14.533537 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f1412083-bc77-401a-b1c1-20c79d85230a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b71e6e57f8f34e9598e0a8c2e2e7a228'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:14.534161 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f1412083-bc77-401a-b1c1-20c79d85230a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b71e6e57f8f34e9598e0a8c2e2e7a228'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:14.536319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:14.542936 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f1412083-bc77-401a-b1c1-20c79d85230a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b71e6e57f8f34e9598e0a8c2e2e7a228'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:14.559120 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-f1412083-bc77-401a-b1c1-20c79d85230a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b71e6e57f8f34e9598e0a8c2e2e7a228'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:14.565341 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1426/2823] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:33:14 2022] DELETE /identity/v3/users/b71e6e57f8f34e9598e0a8c2e2e7a228 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:15.040462 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d5f95-3060-4108-8fee-0f70a4551258 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:15.042021 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-439d5f95-3060-4108-8fee-0f70a4551258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:15.055695 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d5f95-3060-4108-8fee-0f70a4551258 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:15.119670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b644f51-7067-46b0-880d-6cb5cf10b009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:15.120296 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b644f51-7067-46b0-880d-6cb5cf10b009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:15.120634 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b644f51-7067-46b0-880d-6cb5cf10b009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:15.120886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b644f51-7067-46b0-880d-6cb5cf10b009 admin admin] PATH_INFO: `/v3/projects/a94eb6e0b204429e83ad3480585d9eac` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:15.122022 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b644f51-7067-46b0-880d-6cb5cf10b009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a94eb6e0b204429e83ad3480585d9eac)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:15.123911 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b644f51-7067-46b0-880d-6cb5cf10b009 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:15.247699 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:15.261034 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4b644f51-7067-46b0-880d-6cb5cf10b009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a94eb6e0b204429e83ad3480585d9eac', 'request_id': 'req-4b644f51-7067-46b0-880d-6cb5cf10b009'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:15.267050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4b644f51-7067-46b0-880d-6cb5cf10b009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a94eb6e0b204429e83ad3480585d9eac', 'request_id': 'req-4b644f51-7067-46b0-880d-6cb5cf10b009'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:15.274796 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4b644f51-7067-46b0-880d-6cb5cf10b009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a94eb6e0b204429e83ad3480585d9eac', 'request_id': 'req-4b644f51-7067-46b0-880d-6cb5cf10b009'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:15.367610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4b644f51-7067-46b0-880d-6cb5cf10b009 admin admin] The token cache is being invalidate because project a94eb6e0b204429e83ad3480585d9eac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:33:15.367827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4b644f51-7067-46b0-880d-6cb5cf10b009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:15.368436 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4b644f51-7067-46b0-880d-6cb5cf10b009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:15.370711 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1398/2824] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:33:15 2022] DELETE /identity/v3/projects/a94eb6e0b204429e83ad3480585d9eac => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:15.375336 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6a2b3-cb67-4c09-99e5-fc3171da0502 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:15.375792 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6a2b3-cb67-4c09-99e5-fc3171da0502 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:15.375910 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6a2b3-cb67-4c09-99e5-fc3171da0502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:15.402826 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-49f6a2b3-cb67-4c09-99e5-fc3171da0502 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:15.405460 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-49f6a2b3-cb67-4c09-99e5-fc3171da0502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:15.469957 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1427/2825] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:15.482561 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbb28b-884f-479b-8a05-e0992950653c None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:15.482894 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbb28b-884f-479b-8a05-e0992950653c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:15.483235 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbb28b-884f-479b-8a05-e0992950653c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:15.503914 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-5adbb28b-884f-479b-8a05-e0992950653c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:15.505261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-5adbb28b-884f-479b-8a05-e0992950653c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:15.514084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1399/2826] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:15.518426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6a2b3-cb67-4c09-99e5-fc3171da0502 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:15.530566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6a2b3-cb67-4c09-99e5-fc3171da0502 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:15.535458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acddd4a4-8a06-40b8-9f33-c6853129e698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:15.544289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddd4a4-8a06-40b8-9f33-c6853129e698 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:15.544289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddd4a4-8a06-40b8-9f33-c6853129e698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:15.544289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddd4a4-8a06-40b8-9f33-c6853129e698 admin admin] PATH_INFO: `/v3/domains` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:15.544289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acddd4a4-8a06-40b8-9f33-c6853129e698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81794) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:33:15.544289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acddd4a4-8a06-40b8-9f33-c6853129e698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:15.544289 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acddd4a4-8a06-40b8-9f33-c6853129e698 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:15.551199 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1428/2827] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:33:15.555587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbb28b-884f-479b-8a05-e0992950653c None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:15.565501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbb28b-884f-479b-8a05-e0992950653c None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:15.570170 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88917f3d-c620-4250-8f09-e47a341c7f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:15.570759 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88917f3d-c620-4250-8f09-e47a341c7f12 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:15.571042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88917f3d-c620-4250-8f09-e47a341c7f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:15.571315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88917f3d-c620-4250-8f09-e47a341c7f12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:15.571946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88917f3d-c620-4250-8f09-e47a341c7f12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:15.574696 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88917f3d-c620-4250-8f09-e47a341c7f12 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:15.703499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1400/2828] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:33:15 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:33:15.711094 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acddd4a4-8a06-40b8-9f33-c6853129e698 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:15.718271 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acddd4a4-8a06-40b8-9f33-c6853129e698 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:15.775521 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834ff2f-0b00-4633-9e61-d7547fff8783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:15.776179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834ff2f-0b00-4633-9e61-d7547fff8783 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:15.776478 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834ff2f-0b00-4633-9e61-d7547fff8783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:15.776795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834ff2f-0b00-4633-9e61-d7547fff8783 admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:15.777462 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834ff2f-0b00-4633-9e61-d7547fff8783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:15.779628 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834ff2f-0b00-4633-9e61-d7547fff8783 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:15.877565 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1429/2829] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:33:15 2022] POST /identity/v3/users => generated 282 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:33:15.882024 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88917f3d-c620-4250-8f09-e47a341c7f12 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:15.892938 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88917f3d-c620-4250-8f09-e47a341c7f12 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:15.897446 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef53b1-4752-407d-935b-eb966534b1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:15.898050 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef53b1-4752-407d-935b-eb966534b1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:15.898351 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef53b1-4752-407d-935b-eb966534b1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:15.898622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef53b1-4752-407d-935b-eb966534b1b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:15.899288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef53b1-4752-407d-935b-eb966534b1b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:15.901185 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef53b1-4752-407d-935b-eb966534b1b4 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:15.911347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1401/2830] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:33:15.915315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834ff2f-0b00-4633-9e61-d7547fff8783 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:15.925968 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834ff2f-0b00-4633-9e61-d7547fff8783 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:15.930543 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb9f33-b148-4454-9477-8e7a086ec891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:15.931133 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb9f33-b148-4454-9477-8e7a086ec891 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:15.931415 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb9f33-b148-4454-9477-8e7a086ec891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:15.932894 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb9f33-b148-4454-9477-8e7a086ec891 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:15.936109 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb9f33-b148-4454-9477-8e7a086ec891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:15.937741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb9f33-b148-4454-9477-8e7a086ec891 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:15.948625 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1430/2831] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:33:15.953006 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef53b1-4752-407d-935b-eb966534b1b4 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:15.963903 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef53b1-4752-407d-935b-eb966534b1b4 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:15.969259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2bd7b-245e-499d-afb1-35347ae76276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:15.969909 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2bd7b-245e-499d-afb1-35347ae76276 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:15.970219 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2bd7b-245e-499d-afb1-35347ae76276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:15.970492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2bd7b-245e-499d-afb1-35347ae76276 admin admin] PATH_INFO: `/v3/projects/de9aab78b1ac4cba9f869c1e2b28d94f/users/f9465f9a9d144605b4e38097a9d0196c/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:15.986060 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2bd7b-245e-499d-afb1-35347ae76276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de9aab78b1ac4cba9f869c1e2b28d94f, user_id=f9465f9a9d144605b4e38097a9d0196c, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:15.988694 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2bd7b-245e-499d-afb1-35347ae76276 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:16.054077 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1402/2832] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:33:15 2022] PUT /identity/v3/projects/de9aab78b1ac4cba9f869c1e2b28d94f/users/f9465f9a9d144605b4e38097a9d0196c/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:16.065209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbbff9-9a8f-447c-8791-cfc1a0474c9b None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:16.065537 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbbff9-9a8f-447c-8791-cfc1a0474c9b None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:16.065820 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbbff9-9a8f-447c-8791-cfc1a0474c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:16.093705 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-53cbbff9-9a8f-447c-8791-cfc1a0474c9b None None] MFA Rules not processed for user `f9465f9a9d144605b4e38097a9d0196c`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:16.094921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-53cbbff9-9a8f-447c-8791-cfc1a0474c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:16.125200 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:16.162848 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1431/2833] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:16.170827 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2bd7b-245e-499d-afb1-35347ae76276 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:16.231276 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2bd7b-245e-499d-afb1-35347ae76276 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:16.235922 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ea024-8196-4e5b-aabf-11df16815992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:16.236532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ea024-8196-4e5b-aabf-11df16815992 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:16.236832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ea024-8196-4e5b-aabf-11df16815992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:16.237107 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ea024-8196-4e5b-aabf-11df16815992 admin admin] PATH_INFO: `/v3/projects` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:16.237718 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ea024-8196-4e5b-aabf-11df16815992 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:16.239597 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ea024-8196-4e5b-aabf-11df16815992 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:16.335168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1403/2834] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:33:16 2022] POST /identity/v3/projects => generated 363 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:33:16.340241 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cbbff9-9a8f-447c-8791-cfc1a0474c9b None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:16.351014 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cbbff9-9a8f-447c-8791-cfc1a0474c9b None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:16.411759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31853fab-6ea7-4a9b-a566-8f9d6729436a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:16.412433 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31853fab-6ea7-4a9b-a566-8f9d6729436a admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:16.412624 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31853fab-6ea7-4a9b-a566-8f9d6729436a admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:16.412993 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31853fab-6ea7-4a9b-a566-8f9d6729436a admin admin] PATH_INFO: `/v3/users` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:16.413564 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31853fab-6ea7-4a9b-a566-8f9d6729436a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:16.415488 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31853fab-6ea7-4a9b-a566-8f9d6729436a admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:16.529961 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1432/2835] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:33:16 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:33:16.534140 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ea024-8196-4e5b-aabf-11df16815992 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:16.544287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ea024-8196-4e5b-aabf-11df16815992 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:16.548797 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c830cb-ea30-4c5a-ab5b-ff6624a67162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:16.549383 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c830cb-ea30-4c5a-ab5b-ff6624a67162 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:16.549666 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c830cb-ea30-4c5a-ab5b-ff6624a67162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:16.549949 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c830cb-ea30-4c5a-ab5b-ff6624a67162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:16.550587 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c830cb-ea30-4c5a-ab5b-ff6624a67162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:16.552159 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c830cb-ea30-4c5a-ab5b-ff6624a67162 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:16.562576 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1404/2836] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:33:16.566621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31853fab-6ea7-4a9b-a566-8f9d6729436a admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:16.577125 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31853fab-6ea7-4a9b-a566-8f9d6729436a admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:16.581735 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f743590-225c-4130-a49f-84081c6dd166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:16.582370 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f743590-225c-4130-a49f-84081c6dd166 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:16.582650 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f743590-225c-4130-a49f-84081c6dd166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:16.582920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f743590-225c-4130-a49f-84081c6dd166 admin admin] PATH_INFO: `/v3/projects/fda655aefe554cdc9b7a3740ab6de300/users/efe2cfcd351148ed9aefccab3e9f0083/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:16.598541 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f743590-225c-4130-a49f-84081c6dd166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fda655aefe554cdc9b7a3740ab6de300, user_id=efe2cfcd351148ed9aefccab3e9f0083, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:16.601202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f743590-225c-4130-a49f-84081c6dd166 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:16.723342 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1433/2837] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:33:16 2022] PUT /identity/v3/projects/fda655aefe554cdc9b7a3740ab6de300/users/efe2cfcd351148ed9aefccab3e9f0083/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:33:16.727308 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c830cb-ea30-4c5a-ab5b-ff6624a67162 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:16.737372 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c830cb-ea30-4c5a-ab5b-ff6624a67162 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:16.764660 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:16.795000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ab560-1e23-46ef-8f25-75fd13837f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:16.795600 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ab560-1e23-46ef-8f25-75fd13837f0e admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:16.795879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ab560-1e23-46ef-8f25-75fd13837f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:16.796155 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ab560-1e23-46ef-8f25-75fd13837f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:16.796830 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ab560-1e23-46ef-8f25-75fd13837f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:16.798408 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ab560-1e23-46ef-8f25-75fd13837f0e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:16.808381 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1405/2838] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:33:16.812518 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f743590-225c-4130-a49f-84081c6dd166 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:16.824540 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f743590-225c-4130-a49f-84081c6dd166 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:16.830299 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5f7c4-0e3f-4831-bc40-2518b1c49912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:16.834285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5f7c4-0e3f-4831-bc40-2518b1c49912 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:16.834285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5f7c4-0e3f-4831-bc40-2518b1c49912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:16.834285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5f7c4-0e3f-4831-bc40-2518b1c49912 admin admin] PATH_INFO: `/v3/projects/fda655aefe554cdc9b7a3740ab6de300/users/efe2cfcd351148ed9aefccab3e9f0083/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:16.834285 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5f7c4-0e3f-4831-bc40-2518b1c49912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fda655aefe554cdc9b7a3740ab6de300, user_id=efe2cfcd351148ed9aefccab3e9f0083, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:16.836138 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5f7c4-0e3f-4831-bc40-2518b1c49912 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:16.847990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1434/2839] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:33:16 2022] PUT /identity/v3/projects/fda655aefe554cdc9b7a3740ab6de300/users/efe2cfcd351148ed9aefccab3e9f0083/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:16.859434 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6a820-c482-4d0e-91a2-b5f707253f09 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:16.859724 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6a820-c482-4d0e-91a2-b5f707253f09 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:16.860013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6a820-c482-4d0e-91a2-b5f707253f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:16.888080 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-0ab6a820-c482-4d0e-91a2-b5f707253f09 None None] MFA Rules not processed for user `efe2cfcd351148ed9aefccab3e9f0083`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:16.889312 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-0ab6a820-c482-4d0e-91a2-b5f707253f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:16.956746 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1406/2840] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:16.969542 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5f7c4-0e3f-4831-bc40-2518b1c49912 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:16.971292 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-abc5f7c4-0e3f-4831-bc40-2518b1c49912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:17.041101 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5f7c4-0e3f-4831-bc40-2518b1c49912 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:17.044990 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc62082-e767-4852-bab2-133836b9126a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:17.045606 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc62082-e767-4852-bab2-133836b9126a service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:17.045889 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc62082-e767-4852-bab2-133836b9126a service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:17.046167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc62082-e767-4852-bab2-133836b9126a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:17.057630 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc62082-e767-4852-bab2-133836b9126a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:17.059568 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc62082-e767-4852-bab2-133836b9126a service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:17.071716 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1435/2841] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:33:18.019622 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6a820-c482-4d0e-91a2-b5f707253f09 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:18.093519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6a820-c482-4d0e-91a2-b5f707253f09 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:18.098254 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fd8f9-fc4b-492c-ba17-a8dbc0254b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:18.098946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fd8f9-fc4b-492c-ba17-a8dbc0254b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:18.099238 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fd8f9-fc4b-492c-ba17-a8dbc0254b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:18.099515 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fd8f9-fc4b-492c-ba17-a8dbc0254b38 admin admin] PATH_INFO: `/v3/users/f9465f9a9d144605b4e38097a9d0196c` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:18.100658 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841fd8f9-fc4b-492c-ba17-a8dbc0254b38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9465f9a9d144605b4e38097a9d0196c)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:18.102578 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841fd8f9-fc4b-492c-ba17-a8dbc0254b38 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:18.353306 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-841fd8f9-fc4b-492c-ba17-a8dbc0254b38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9465f9a9d144605b4e38097a9d0196c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:18.354151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-841fd8f9-fc4b-492c-ba17-a8dbc0254b38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9465f9a9d144605b4e38097a9d0196c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:18.358281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:18.488099 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-841fd8f9-fc4b-492c-ba17-a8dbc0254b38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9465f9a9d144605b4e38097a9d0196c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:18.501194 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-841fd8f9-fc4b-492c-ba17-a8dbc0254b38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9465f9a9d144605b4e38097a9d0196c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:18.508042 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-841fd8f9-fc4b-492c-ba17-a8dbc0254b38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9465f9a9d144605b4e38097a9d0196c'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:18.514762 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1407/2842] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:33:18 2022] DELETE /identity/v3/users/f9465f9a9d144605b4e38097a9d0196c => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:18.519201 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9b186-6cbb-4e42-8ae2-6efcfd8932e0 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:18.519671 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9b186-6cbb-4e42-8ae2-6efcfd8932e0 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:18.519773 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9b186-6cbb-4e42-8ae2-6efcfd8932e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:18.540209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-eda9b186-6cbb-4e42-8ae2-6efcfd8932e0 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:18.541920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-eda9b186-6cbb-4e42-8ae2-6efcfd8932e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:18.561903 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:18.603064 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1436/2843] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:18.613123 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fd8f9-fc4b-492c-ba17-a8dbc0254b38 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:18.614560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-841fd8f9-fc4b-492c-ba17-a8dbc0254b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:18.673881 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fd8f9-fc4b-492c-ba17-a8dbc0254b38 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:18.677698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e8ff1-782f-4ce2-b264-f13b613a306b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:18.678324 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e8ff1-782f-4ce2-b264-f13b613a306b service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:18.678584 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e8ff1-782f-4ce2-b264-f13b613a306b service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:18.678856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e8ff1-782f-4ce2-b264-f13b613a306b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:18.687775 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e8ff1-782f-4ce2-b264-f13b613a306b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:18.689272 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e8ff1-782f-4ce2-b264-f13b613a306b service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:18.698519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1408/2844] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 12 12:33:19.142777 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9b186-6cbb-4e42-8ae2-6efcfd8932e0 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:19.144273 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-eda9b186-6cbb-4e42-8ae2-6efcfd8932e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:19.156986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9b186-6cbb-4e42-8ae2-6efcfd8932e0 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:19.161652 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:19.162333 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:19.162615 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:19.162951 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae admin admin] PATH_INFO: `/v3/projects/de9aab78b1ac4cba9f869c1e2b28d94f` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:19.164210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae admin admin] RBAC: Authorizing `identity:delete_project(project_id=de9aab78b1ac4cba9f869c1e2b28d94f)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:19.166863 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:19.265857 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de9aab78b1ac4cba9f869c1e2b28d94f', 'request_id': 'req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:19.271213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de9aab78b1ac4cba9f869c1e2b28d94f', 'request_id': 'req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:19.277375 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de9aab78b1ac4cba9f869c1e2b28d94f', 'request_id': 'req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:19.344986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae admin admin] The token cache is being invalidate because project de9aab78b1ac4cba9f869c1e2b28d94f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:33:19.345396 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:19.345895 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:19.347797 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1437/2845] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:33:19 2022] DELETE /identity/v3/projects/de9aab78b1ac4cba9f869c1e2b28d94f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:19.352153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e8ff1-782f-4ce2-b264-f13b613a306b service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:19.362808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e8ff1-782f-4ce2-b264-f13b613a306b service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:19.419786 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea826105-48f7-460e-a07d-0dcba902f8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:19.420402 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea826105-48f7-460e-a07d-0dcba902f8ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:19.420712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea826105-48f7-460e-a07d-0dcba902f8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:19.420986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea826105-48f7-460e-a07d-0dcba902f8ac admin admin] PATH_INFO: `/v3/users/efe2cfcd351148ed9aefccab3e9f0083` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:19.422087 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea826105-48f7-460e-a07d-0dcba902f8ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=efe2cfcd351148ed9aefccab3e9f0083)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:19.424406 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea826105-48f7-460e-a07d-0dcba902f8ac admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:19.667623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ea826105-48f7-460e-a07d-0dcba902f8ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efe2cfcd351148ed9aefccab3e9f0083'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:19.668287 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ea826105-48f7-460e-a07d-0dcba902f8ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efe2cfcd351148ed9aefccab3e9f0083'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:19.734222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ea826105-48f7-460e-a07d-0dcba902f8ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efe2cfcd351148ed9aefccab3e9f0083'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:19.747883 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ea826105-48f7-460e-a07d-0dcba902f8ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efe2cfcd351148ed9aefccab3e9f0083'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:19.749891 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:19.755229 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-ea826105-48f7-460e-a07d-0dcba902f8ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efe2cfcd351148ed9aefccab3e9f0083'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:19.761288 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1409/2846] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:33:19 2022] DELETE /identity/v3/users/efe2cfcd351148ed9aefccab3e9f0083 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:20.153382 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:20.160338 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:20.176175 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae84e62-099b-4d94-b5b8-77ca7fc35eae admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:20.252314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0ecae-70c5-4286-a3e5-2d44b205bc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:20.252314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0ecae-70c5-4286-a3e5-2d44b205bc10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:20.252314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0ecae-70c5-4286-a3e5-2d44b205bc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:20.252314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0ecae-70c5-4286-a3e5-2d44b205bc10 admin admin] PATH_INFO: `/v3/projects/fda655aefe554cdc9b7a3740ab6de300` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:20.252741 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0ecae-70c5-4286-a3e5-2d44b205bc10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fda655aefe554cdc9b7a3740ab6de300)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:20.253591 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0ecae-70c5-4286-a3e5-2d44b205bc10 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:20.381820 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3be0ecae-70c5-4286-a3e5-2d44b205bc10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fda655aefe554cdc9b7a3740ab6de300', 'request_id': 'req-3be0ecae-70c5-4286-a3e5-2d44b205bc10'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:20.383008 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:20.389763 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3be0ecae-70c5-4286-a3e5-2d44b205bc10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fda655aefe554cdc9b7a3740ab6de300', 'request_id': 'req-3be0ecae-70c5-4286-a3e5-2d44b205bc10'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:20.401856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3be0ecae-70c5-4286-a3e5-2d44b205bc10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fda655aefe554cdc9b7a3740ab6de300', 'request_id': 'req-3be0ecae-70c5-4286-a3e5-2d44b205bc10'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:20.500071 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3be0ecae-70c5-4286-a3e5-2d44b205bc10 admin admin] The token cache is being invalidate because project fda655aefe554cdc9b7a3740ab6de300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:33:20.500502 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3be0ecae-70c5-4286-a3e5-2d44b205bc10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:20.501254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-3be0ecae-70c5-4286-a3e5-2d44b205bc10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:20.506155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1438/2847] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:33:20 2022] DELETE /identity/v3/projects/fda655aefe554cdc9b7a3740ab6de300 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:20.509841 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b96bf-e91a-4eb4-850a-e656c616fea3 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:20.510341 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b96bf-e91a-4eb4-850a-e656c616fea3 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:20.510999 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b96bf-e91a-4eb4-850a-e656c616fea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:20.531236 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-4e2b96bf-e91a-4eb4-850a-e656c616fea3 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:20.538742 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4e2b96bf-e91a-4eb4-850a-e656c616fea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:20.599259 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1410/2848] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:20.607315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aafa25-db07-4231-b793-9bb97ccda8f0 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:20.607602 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aafa25-db07-4231-b793-9bb97ccda8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:20.607879 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aafa25-db07-4231-b793-9bb97ccda8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:20.628276 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-09aafa25-db07-4231-b793-9bb97ccda8f0 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:20.629599 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-09aafa25-db07-4231-b793-9bb97ccda8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:20.638267 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1439/2849] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:20.642450 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b96bf-e91a-4eb4-850a-e656c616fea3 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:20.653436 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b96bf-e91a-4eb4-850a-e656c616fea3 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:20.658565 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc1adf-bf92-4911-90b5-f246a142f4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:20.659202 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc1adf-bf92-4911-90b5-f246a142f4cf admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:20.659401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc1adf-bf92-4911-90b5-f246a142f4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:20.659677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc1adf-bf92-4911-90b5-f246a142f4cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:20.660572 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc1adf-bf92-4911-90b5-f246a142f4cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:33:20.660939 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc1adf-bf92-4911-90b5-f246a142f4cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:20.662758 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc1adf-bf92-4911-90b5-f246a142f4cf admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:20.679866 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1411/2850] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:33:20.681227 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aafa25-db07-4231-b793-9bb97ccda8f0 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:20.691188 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aafa25-db07-4231-b793-9bb97ccda8f0 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:20.696168 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d089304-15ad-4168-aa2d-37cd461fe4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:20.696950 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d089304-15ad-4168-aa2d-37cd461fe4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:20.697319 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d089304-15ad-4168-aa2d-37cd461fe4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:20.697679 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d089304-15ad-4168-aa2d-37cd461fe4e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:20.698411 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d089304-15ad-4168-aa2d-37cd461fe4e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:20.700608 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d089304-15ad-4168-aa2d-37cd461fe4e0 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:20.819772 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1440/2851] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:33:20 2022] POST /identity/v3/projects => generated 369 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:33:20.823793 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc1adf-bf92-4911-90b5-f246a142f4cf admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:20.835245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc1adf-bf92-4911-90b5-f246a142f4cf admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:20.895654 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d9396-e6c9-4ad8-a616-d1dc22bb3794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:20.896255 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d9396-e6c9-4ad8-a616-d1dc22bb3794 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:20.896532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d9396-e6c9-4ad8-a616-d1dc22bb3794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:20.896832 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d9396-e6c9-4ad8-a616-d1dc22bb3794 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:20.897467 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d9396-e6c9-4ad8-a616-d1dc22bb3794 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:20.899346 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d9396-e6c9-4ad8-a616-d1dc22bb3794 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:20.980084 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1412/2852] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:33:20 2022] POST /identity/v3/users => generated 285 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:33:20.984671 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d089304-15ad-4168-aa2d-37cd461fe4e0 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:20.996090 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d089304-15ad-4168-aa2d-37cd461fe4e0 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:21.001003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d94a83-0e37-4065-8550-3f4d5c4aa7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:21.001598 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d94a83-0e37-4065-8550-3f4d5c4aa7aa admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:21.001879 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d94a83-0e37-4065-8550-3f4d5c4aa7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:21.002154 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d94a83-0e37-4065-8550-3f4d5c4aa7aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:21.002807 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d94a83-0e37-4065-8550-3f4d5c4aa7aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:21.004386 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d94a83-0e37-4065-8550-3f4d5c4aa7aa admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:21.016532 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1441/2853] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:33:21.056230 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d9396-e6c9-4ad8-a616-d1dc22bb3794 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:21.067086 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d9396-e6c9-4ad8-a616-d1dc22bb3794 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:21.072149 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7962360-50c1-46bb-9f7d-0b2e5b9eed43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:21.073560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7962360-50c1-46bb-9f7d-0b2e5b9eed43 admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:21.074274 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7962360-50c1-46bb-9f7d-0b2e5b9eed43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:21.074712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7962360-50c1-46bb-9f7d-0b2e5b9eed43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:21.075506 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7962360-50c1-46bb-9f7d-0b2e5b9eed43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:21.077425 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7962360-50c1-46bb-9f7d-0b2e5b9eed43 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:21.099103 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1413/2854] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 12 12:33:21.100378 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d94a83-0e37-4065-8550-3f4d5c4aa7aa admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:21.110692 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d94a83-0e37-4065-8550-3f4d5c4aa7aa admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:21.115320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955d035-587d-46ce-a09e-98e0abeac1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:21.115988 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955d035-587d-46ce-a09e-98e0abeac1f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:21.116315 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955d035-587d-46ce-a09e-98e0abeac1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:21.116593 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955d035-587d-46ce-a09e-98e0abeac1f2 admin admin] PATH_INFO: `/v3/projects/3af6c6687c5947588cf2c28ff5ac35dc/users/bb5d09689f0a400ebc1fc90ba16c7b70/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:21.131180 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955d035-587d-46ce-a09e-98e0abeac1f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3af6c6687c5947588cf2c28ff5ac35dc, user_id=bb5d09689f0a400ebc1fc90ba16c7b70, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:21.133893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955d035-587d-46ce-a09e-98e0abeac1f2 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:21.273747 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1442/2855] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:33:21 2022] PUT /identity/v3/projects/3af6c6687c5947588cf2c28ff5ac35dc/users/bb5d09689f0a400ebc1fc90ba16c7b70/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:33:21.283555 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdde08-b431-41b0-a54f-a1a8351bd220 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:21.283841 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdde08-b431-41b0-a54f-a1a8351bd220 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:21.284108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdde08-b431-41b0-a54f-a1a8351bd220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:21.317210 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-defdde08-b431-41b0-a54f-a1a8351bd220 None None] MFA Rules not processed for user `bb5d09689f0a400ebc1fc90ba16c7b70`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:21.317210 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-defdde08-b431-41b0-a54f-a1a8351bd220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:21.361826 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:21.381604 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1414/2856] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:21.386933 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955d035-587d-46ce-a09e-98e0abeac1f2 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:21.456526 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955d035-587d-46ce-a09e-98e0abeac1f2 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:21.461259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546a86c-3022-4613-a622-c97631e79f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:21.462179 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546a86c-3022-4613-a622-c97631e79f86 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:21.462460 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546a86c-3022-4613-a622-c97631e79f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:21.462734 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546a86c-3022-4613-a622-c97631e79f86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:21.463364 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546a86c-3022-4613-a622-c97631e79f86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:21.465597 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546a86c-3022-4613-a622-c97631e79f86 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:21.541687 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1443/2857] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:33:21 2022] POST /identity/v3/projects => generated 369 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:33:21.545857 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdde08-b431-41b0-a54f-a1a8351bd220 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:21.556910 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdde08-b431-41b0-a54f-a1a8351bd220 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:21.620407 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ce088-5a28-418d-ba46-db3a38d9b317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:21.621083 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ce088-5a28-418d-ba46-db3a38d9b317 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:21.621363 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ce088-5a28-418d-ba46-db3a38d9b317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:21.621632 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ce088-5a28-418d-ba46-db3a38d9b317 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:21.622282 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ce088-5a28-418d-ba46-db3a38d9b317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:21.624409 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ce088-5a28-418d-ba46-db3a38d9b317 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:21.740676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1415/2858] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:33:21 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:33:21.744566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546a86c-3022-4613-a622-c97631e79f86 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:21.755040 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546a86c-3022-4613-a622-c97631e79f86 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:21.759534 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6b2ac-c2cc-4788-bc6e-29e5d74924a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:21.760126 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6b2ac-c2cc-4788-bc6e-29e5d74924a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:21.760935 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6b2ac-c2cc-4788-bc6e-29e5d74924a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:21.761210 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6b2ac-c2cc-4788-bc6e-29e5d74924a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:21.761876 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6b2ac-c2cc-4788-bc6e-29e5d74924a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:21.769237 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6b2ac-c2cc-4788-bc6e-29e5d74924a4 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:21.774034 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1444/2859] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:33:21.779016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ce088-5a28-418d-ba46-db3a38d9b317 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:21.789212 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ce088-5a28-418d-ba46-db3a38d9b317 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:21.794151 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34920c39-3f29-40a6-b5ce-647217f28f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:21.794867 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34920c39-3f29-40a6-b5ce-647217f28f6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:21.795220 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34920c39-3f29-40a6-b5ce-647217f28f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:21.795566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34920c39-3f29-40a6-b5ce-647217f28f6a admin admin] PATH_INFO: `/v3/projects/757579798be94abb90267bdb0e4a06b7/users/187b8fdca28c4212a2b957739c6fedfe/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:21.817215 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34920c39-3f29-40a6-b5ce-647217f28f6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=757579798be94abb90267bdb0e4a06b7, user_id=187b8fdca28c4212a2b957739c6fedfe, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:21.819939 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34920c39-3f29-40a6-b5ce-647217f28f6a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:21.884113 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1416/2860] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:33:21 2022] PUT /identity/v3/projects/757579798be94abb90267bdb0e4a06b7/users/187b8fdca28c4212a2b957739c6fedfe/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:21.889043 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6b2ac-c2cc-4788-bc6e-29e5d74924a4 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:21.899086 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6b2ac-c2cc-4788-bc6e-29e5d74924a4 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:21.945431 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:21.957898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ea5f7-366a-4e4c-aa61-070e8006ca5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:21.958507 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea5f7-366a-4e4c-aa61-070e8006ca5a admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:21.958791 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea5f7-366a-4e4c-aa61-070e8006ca5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:21.959071 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea5f7-366a-4e4c-aa61-070e8006ca5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:21.959719 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ea5f7-366a-4e4c-aa61-070e8006ca5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:21.961750 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ea5f7-366a-4e4c-aa61-070e8006ca5a admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:21.972912 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1445/2861] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:33:21.983698 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34920c39-3f29-40a6-b5ce-647217f28f6a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:21.987778 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34920c39-3f29-40a6-b5ce-647217f28f6a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:21.992967 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5449b-52d3-488d-a991-667e0d9bf8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:21.993615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5449b-52d3-488d-a991-667e0d9bf8d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:21.998297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5449b-52d3-488d-a991-667e0d9bf8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:21.998297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5449b-52d3-488d-a991-667e0d9bf8d9 admin admin] PATH_INFO: `/v3/projects/757579798be94abb90267bdb0e4a06b7/users/187b8fdca28c4212a2b957739c6fedfe/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:21.998297 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5449b-52d3-488d-a991-667e0d9bf8d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=757579798be94abb90267bdb0e4a06b7, user_id=187b8fdca28c4212a2b957739c6fedfe, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:21.998985 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5449b-52d3-488d-a991-667e0d9bf8d9 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:22.009725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1417/2862] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:33:21 2022] PUT /identity/v3/projects/757579798be94abb90267bdb0e4a06b7/users/187b8fdca28c4212a2b957739c6fedfe/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:33:22.081194 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d87f5e-502b-42ac-9887-738772ae5c78 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:22.081530 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d87f5e-502b-42ac-9887-738772ae5c78 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:22.081808 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d87f5e-502b-42ac-9887-738772ae5c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:22.111073 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-e0d87f5e-502b-42ac-9887-738772ae5c78 None None] MFA Rules not processed for user `187b8fdca28c4212a2b957739c6fedfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:22.112292 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e0d87f5e-502b-42ac-9887-738772ae5c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:22.190099 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1446/2863] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:22.203243 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5449b-52d3-488d-a991-667e0d9bf8d9 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:22.205073 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1ef5449b-52d3-488d-a991-667e0d9bf8d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:22.271794 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5449b-52d3-488d-a991-667e0d9bf8d9 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:22.276116 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88ae2a-1be1-4221-aa4b-a63d1d2bc5dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:22.276681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88ae2a-1be1-4221-aa4b-a63d1d2bc5dd service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:22.279492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88ae2a-1be1-4221-aa4b-a63d1d2bc5dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:22.279492 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88ae2a-1be1-4221-aa4b-a63d1d2bc5dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:22.288501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88ae2a-1be1-4221-aa4b-a63d1d2bc5dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:22.289963 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88ae2a-1be1-4221-aa4b-a63d1d2bc5dd service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:22.301358 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1418/2864] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:33:22.429934 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d87f5e-502b-42ac-9887-738772ae5c78 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:22.431459 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-e0d87f5e-502b-42ac-9887-738772ae5c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:22.445627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d87f5e-502b-42ac-9887-738772ae5c78 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:22.447447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7169bec-c600-49e8-949c-54624fa5e923 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:22.448715 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7169bec-c600-49e8-949c-54624fa5e923 service cinder] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:22.448998 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7169bec-c600-49e8-949c-54624fa5e923 service cinder] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:22.449288 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7169bec-c600-49e8-949c-54624fa5e923 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:22.506928 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7169bec-c600-49e8-949c-54624fa5e923 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:22.508363 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7169bec-c600-49e8-949c-54624fa5e923 service cinder] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:22.517486 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1447/2865] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:33:26.895676 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88ae2a-1be1-4221-aa4b-a63d1d2bc5dd service cinder] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:26.897179 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4a88ae2a-1be1-4221-aa4b-a63d1d2bc5dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:26.944066 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:26.966036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88ae2a-1be1-4221-aa4b-a63d1d2bc5dd service cinder] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:26.970665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025581d-13fb-46ba-bcbf-4082fdcd6680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:26.971281 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025581d-13fb-46ba-bcbf-4082fdcd6680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:26.971559 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025581d-13fb-46ba-bcbf-4082fdcd6680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:26.971828 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025581d-13fb-46ba-bcbf-4082fdcd6680 admin admin] PATH_INFO: `/v3/users/39ef1e3c0af4441c873b0b2d959f7b1b` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:26.973033 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025581d-13fb-46ba-bcbf-4082fdcd6680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39ef1e3c0af4441c873b0b2d959f7b1b)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:26.974921 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025581d-13fb-46ba-bcbf-4082fdcd6680 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:27.171849 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1025581d-13fb-46ba-bcbf-4082fdcd6680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39ef1e3c0af4441c873b0b2d959f7b1b'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:27.172598 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1025581d-13fb-46ba-bcbf-4082fdcd6680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39ef1e3c0af4441c873b0b2d959f7b1b'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:27.248360 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1025581d-13fb-46ba-bcbf-4082fdcd6680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39ef1e3c0af4441c873b0b2d959f7b1b'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:27.261200 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1025581d-13fb-46ba-bcbf-4082fdcd6680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39ef1e3c0af4441c873b0b2d959f7b1b'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:27.268027 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-1025581d-13fb-46ba-bcbf-4082fdcd6680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39ef1e3c0af4441c873b0b2d959f7b1b'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:27.274108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1419/2866] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:33:26 2022] DELETE /identity/v3/users/39ef1e3c0af4441c873b0b2d959f7b1b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:27.278954 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae63584-5ba1-452f-91d6-00784ca0610d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:27.279432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae63584-5ba1-452f-91d6-00784ca0610d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:27.279538 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae63584-5ba1-452f-91d6-00784ca0610d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:27.299256 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-eae63584-5ba1-452f-91d6-00784ca0610d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:27.300457 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-eae63584-5ba1-452f-91d6-00784ca0610d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:27.360994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1448/2867] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:27.370863 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025581d-13fb-46ba-bcbf-4082fdcd6680 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:27.372597 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-1025581d-13fb-46ba-bcbf-4082fdcd6680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:27.429977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025581d-13fb-46ba-bcbf-4082fdcd6680 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:27.433651 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803164e-fdc9-4a54-a769-a6f742e31d7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:27.434328 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803164e-fdc9-4a54-a769-a6f742e31d7e service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:27.434606 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803164e-fdc9-4a54-a769-a6f742e31d7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:27.434923 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803164e-fdc9-4a54-a769-a6f742e31d7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:27.444171 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803164e-fdc9-4a54-a769-a6f742e31d7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:27.445776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803164e-fdc9-4a54-a769-a6f742e31d7e service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:27.455689 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1420/2868] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:33:27.765987 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae63584-5ba1-452f-91d6-00784ca0610d None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:27.767458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-eae63584-5ba1-452f-91d6-00784ca0610d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:27.779890 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae63584-5ba1-452f-91d6-00784ca0610d None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:27.784418 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361ccc7-bc4f-4151-897f-f5f0190cb8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:27.785135 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361ccc7-bc4f-4151-897f-f5f0190cb8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:27.785417 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361ccc7-bc4f-4151-897f-f5f0190cb8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:27.785686 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361ccc7-bc4f-4151-897f-f5f0190cb8d4 admin admin] PATH_INFO: `/v3/users/bb5d09689f0a400ebc1fc90ba16c7b70` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:27.786797 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361ccc7-bc4f-4151-897f-f5f0190cb8d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb5d09689f0a400ebc1fc90ba16c7b70)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:27.788722 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361ccc7-bc4f-4151-897f-f5f0190cb8d4 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:27.791156 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=100 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:27.919089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803164e-fdc9-4a54-a769-a6f742e31d7e service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:27.920542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4803164e-fdc9-4a54-a769-a6f742e31d7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:27.933681 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803164e-fdc9-4a54-a769-a6f742e31d7e service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:27.938399 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223a979-9983-44eb-af55-ca6801e3b24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:27.939025 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223a979-9983-44eb-af55-ca6801e3b24b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:27.939237 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223a979-9983-44eb-af55-ca6801e3b24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:27.939507 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223a979-9983-44eb-af55-ca6801e3b24b admin admin] PATH_INFO: `/v3/projects/938bfb2e36e34b4283988e8b6bd9ee92` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:27.940654 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223a979-9983-44eb-af55-ca6801e3b24b admin admin] RBAC: Authorizing `identity:delete_project(project_id=938bfb2e36e34b4283988e8b6bd9ee92)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:27.942542 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223a979-9983-44eb-af55-ca6801e3b24b admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:28.101708 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-6361ccc7-bc4f-4151-897f-f5f0190cb8d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb5d09689f0a400ebc1fc90ba16c7b70'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:28.106007 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f223a979-9983-44eb-af55-ca6801e3b24b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '938bfb2e36e34b4283988e8b6bd9ee92', 'request_id': 'req-f223a979-9983-44eb-af55-ca6801e3b24b'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:28.111756 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f223a979-9983-44eb-af55-ca6801e3b24b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '938bfb2e36e34b4283988e8b6bd9ee92', 'request_id': 'req-f223a979-9983-44eb-af55-ca6801e3b24b'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:28.112855 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:28.118370 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f223a979-9983-44eb-af55-ca6801e3b24b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '938bfb2e36e34b4283988e8b6bd9ee92', 'request_id': 'req-f223a979-9983-44eb-af55-ca6801e3b24b'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:28.179449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-6361ccc7-bc4f-4151-897f-f5f0190cb8d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb5d09689f0a400ebc1fc90ba16c7b70'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:28.180105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-6361ccc7-bc4f-4151-897f-f5f0190cb8d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb5d09689f0a400ebc1fc90ba16c7b70'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:28.188387 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-6361ccc7-bc4f-4151-897f-f5f0190cb8d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb5d09689f0a400ebc1fc90ba16c7b70'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:28.200654 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-6361ccc7-bc4f-4151-897f-f5f0190cb8d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb5d09689f0a400ebc1fc90ba16c7b70'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:28.206613 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1449/2869] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:33:27 2022] DELETE /identity/v3/users/bb5d09689f0a400ebc1fc90ba16c7b70 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:28.210795 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58007141-e5a0-40d5-9497-b7d3f2686f17 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:28.211079 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58007141-e5a0-40d5-9497-b7d3f2686f17 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:28.211353 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58007141-e5a0-40d5-9497-b7d3f2686f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:28.229881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-58007141-e5a0-40d5-9497-b7d3f2686f17 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:28.231241 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-58007141-e5a0-40d5-9497-b7d3f2686f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:28.281870 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f223a979-9983-44eb-af55-ca6801e3b24b admin admin] The token cache is being invalidate because project 938bfb2e36e34b4283988e8b6bd9ee92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:33:28.282145 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f223a979-9983-44eb-af55-ca6801e3b24b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:28.282779 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-f223a979-9983-44eb-af55-ca6801e3b24b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:28.284263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1421/2870] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:33:27 2022] DELETE /identity/v3/projects/938bfb2e36e34b4283988e8b6bd9ee92 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 12 12:33:28.289270 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1450/2871] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:33:28.299523 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223a979-9983-44eb-af55-ca6801e3b24b admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:28.301125 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-f223a979-9983-44eb-af55-ca6801e3b24b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:28.366850 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223a979-9983-44eb-af55-ca6801e3b24b admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:28.422488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dddc1a-5747-4f00-b51b-75fd81f339a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:28.423111 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dddc1a-5747-4f00-b51b-75fd81f339a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:28.423391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dddc1a-5747-4f00-b51b-75fd81f339a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:28.423665 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dddc1a-5747-4f00-b51b-75fd81f339a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:28.434431 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dddc1a-5747-4f00-b51b-75fd81f339a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:28.436108 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dddc1a-5747-4f00-b51b-75fd81f339a5 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:28.447013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1422/2872] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:33:29.113689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58007141-e5a0-40d5-9497-b7d3f2686f17 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:29.115110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-58007141-e5a0-40d5-9497-b7d3f2686f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:29.127240 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58007141-e5a0-40d5-9497-b7d3f2686f17 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:29.131874 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:29.132560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:29.132856 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:29.133131 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b admin admin] PATH_INFO: `/v3/projects/3af6c6687c5947588cf2c28ff5ac35dc` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:29.134259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3af6c6687c5947588cf2c28ff5ac35dc)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:29.136176 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:29.266487 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3af6c6687c5947588cf2c28ff5ac35dc', 'request_id': 'req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:29.272263 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3af6c6687c5947588cf2c28ff5ac35dc', 'request_id': 'req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:29.279403 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3af6c6687c5947588cf2c28ff5ac35dc', 'request_id': 'req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:29.355560 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b admin admin] The token cache is being invalidate because project 3af6c6687c5947588cf2c28ff5ac35dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:33:29.356209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:29.356551 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:29.358061 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1451/2873] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:33:29 2022] DELETE /identity/v3/projects/3af6c6687c5947588cf2c28ff5ac35dc => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:29.361976 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dddc1a-5747-4f00-b51b-75fd81f339a5 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:29.372211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dddc1a-5747-4f00-b51b-75fd81f339a5 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:29.428051 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521a08a-6eac-458d-be2b-b1140e687d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:29.428693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521a08a-6eac-458d-be2b-b1140e687d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:29.428980 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521a08a-6eac-458d-be2b-b1140e687d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:29.429248 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521a08a-6eac-458d-be2b-b1140e687d7a admin admin] PATH_INFO: `/v3/users/187b8fdca28c4212a2b957739c6fedfe` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:29.430353 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521a08a-6eac-458d-be2b-b1140e687d7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=187b8fdca28c4212a2b957739c6fedfe)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:29.432237 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521a08a-6eac-458d-be2b-b1140e687d7a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:29.650602 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a521a08a-6eac-458d-be2b-b1140e687d7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '187b8fdca28c4212a2b957739c6fedfe'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:29.651247 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a521a08a-6eac-458d-be2b-b1140e687d7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '187b8fdca28c4212a2b957739c6fedfe'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:29.653357 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:29.761205 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a521a08a-6eac-458d-be2b-b1140e687d7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '187b8fdca28c4212a2b957739c6fedfe'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:29.774890 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a521a08a-6eac-458d-be2b-b1140e687d7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '187b8fdca28c4212a2b957739c6fedfe'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:29.786623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-a521a08a-6eac-458d-be2b-b1140e687d7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '187b8fdca28c4212a2b957739c6fedfe'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:29.793070 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1423/2874] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:33:29 2022] DELETE /identity/v3/users/187b8fdca28c4212a2b957739c6fedfe => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:30.454921 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:30.456405 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:30.466627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:30.470793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf6db3-73a9-4d6e-93ff-06b5f0832a8b admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:30.526074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eea170-cf5f-4fb3-97eb-146078f044dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:30.526704 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eea170-cf5f-4fb3-97eb-146078f044dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:30.526989 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eea170-cf5f-4fb3-97eb-146078f044dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:30.527267 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eea170-cf5f-4fb3-97eb-146078f044dc admin admin] PATH_INFO: `/v3/projects/757579798be94abb90267bdb0e4a06b7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:30.529155 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eea170-cf5f-4fb3-97eb-146078f044dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=757579798be94abb90267bdb0e4a06b7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:30.531068 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eea170-cf5f-4fb3-97eb-146078f044dc admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:30.666213 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-44eea170-cf5f-4fb3-97eb-146078f044dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '757579798be94abb90267bdb0e4a06b7', 'request_id': 'req-44eea170-cf5f-4fb3-97eb-146078f044dc'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:30.671972 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-44eea170-cf5f-4fb3-97eb-146078f044dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '757579798be94abb90267bdb0e4a06b7', 'request_id': 'req-44eea170-cf5f-4fb3-97eb-146078f044dc'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:30.678573 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-44eea170-cf5f-4fb3-97eb-146078f044dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '757579798be94abb90267bdb0e4a06b7', 'request_id': 'req-44eea170-cf5f-4fb3-97eb-146078f044dc'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:30.807782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-44eea170-cf5f-4fb3-97eb-146078f044dc admin admin] The token cache is being invalidate because project 757579798be94abb90267bdb0e4a06b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:33:30.807994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-44eea170-cf5f-4fb3-97eb-146078f044dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:30.808617 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-44eea170-cf5f-4fb3-97eb-146078f044dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:30.810252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1452/2875] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:33:30 2022] DELETE /identity/v3/projects/757579798be94abb90267bdb0e4a06b7 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:30.815163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136b6a8-3ea2-4103-ad46-2706236fac26 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:30.815670 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136b6a8-3ea2-4103-ad46-2706236fac26 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:30.815791 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136b6a8-3ea2-4103-ad46-2706236fac26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:30.836959 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-9136b6a8-3ea2-4103-ad46-2706236fac26 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:30.838160 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9136b6a8-3ea2-4103-ad46-2706236fac26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:30.904912 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1424/2876] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:33:30.912552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235cecf-e703-4137-b85c-3635be363f3d None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:30.912893 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235cecf-e703-4137-b85c-3635be363f3d None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:30.913167 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235cecf-e703-4137-b85c-3635be363f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:30.932943 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-b235cecf-e703-4137-b85c-3635be363f3d None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:30.934143 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-b235cecf-e703-4137-b85c-3635be363f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:30.942093 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1453/2877] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:30.946222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136b6a8-3ea2-4103-ad46-2706236fac26 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:30.956813 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136b6a8-3ea2-4103-ad46-2706236fac26 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:30.961396 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3308012-2960-44f5-8367-bb267eb8ff7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:30.962249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3308012-2960-44f5-8367-bb267eb8ff7b admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:30.962529 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3308012-2960-44f5-8367-bb267eb8ff7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:30.962798 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3308012-2960-44f5-8367-bb267eb8ff7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:30.963499 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3308012-2960-44f5-8367-bb267eb8ff7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81793) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 12 12:33:30.963799 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3308012-2960-44f5-8367-bb267eb8ff7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:30.966263 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3308012-2960-44f5-8367-bb267eb8ff7b admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:30.977992 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1425/2878] 10.222.0.27 () {58 vars in 1153 bytes} [Fri Aug 12 12:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 12 12:33:30.982233 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235cecf-e703-4137-b85c-3635be363f3d None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:30.991544 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235cecf-e703-4137-b85c-3635be363f3d None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:30.996130 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4c39b-afd4-4e09-a87e-dd8c95aaf0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:30.996788 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4c39b-afd4-4e09-a87e-dd8c95aaf0d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:30.997067 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4c39b-afd4-4e09-a87e-dd8c95aaf0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:30.997432 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4c39b-afd4-4e09-a87e-dd8c95aaf0d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:30.998059 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4c39b-afd4-4e09-a87e-dd8c95aaf0d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:31.000428 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4c39b-afd4-4e09-a87e-dd8c95aaf0d8 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:31.078195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1454/2879] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:33:30 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:33:31.082339 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3308012-2960-44f5-8367-bb267eb8ff7b admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:31.092798 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3308012-2960-44f5-8367-bb267eb8ff7b admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:31.148096 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc4940-6bd3-47e0-9761-049fb2b12c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:31.148725 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc4940-6bd3-47e0-9761-049fb2b12c0a admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:31.149016 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc4940-6bd3-47e0-9761-049fb2b12c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:31.149295 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc4940-6bd3-47e0-9761-049fb2b12c0a admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:31.149993 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc4940-6bd3-47e0-9761-049fb2b12c0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:31.151977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc4940-6bd3-47e0-9761-049fb2b12c0a admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:31.248184 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1426/2880] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:33:31 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:33:31.252304 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4c39b-afd4-4e09-a87e-dd8c95aaf0d8 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:31.263342 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4c39b-afd4-4e09-a87e-dd8c95aaf0d8 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:31.268298 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ab497-9d38-4cc3-a093-0e8a9ae17e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:31.268917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ab497-9d38-4cc3-a093-0e8a9ae17e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:31.269195 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ab497-9d38-4cc3-a093-0e8a9ae17e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:31.269621 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ab497-9d38-4cc3-a093-0e8a9ae17e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:31.270254 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ab497-9d38-4cc3-a093-0e8a9ae17e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:31.271846 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ab497-9d38-4cc3-a093-0e8a9ae17e4b admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:31.282314 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1455/2881] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:33:31.286610 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc4940-6bd3-47e0-9761-049fb2b12c0a admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:31.297210 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc4940-6bd3-47e0-9761-049fb2b12c0a admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:31.301968 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574ef34-e74a-42e0-a2f8-6b3089b9a02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:31.302620 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574ef34-e74a-42e0-a2f8-6b3089b9a02e admin admin] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:31.302935 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574ef34-e74a-42e0-a2f8-6b3089b9a02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:31.303208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574ef34-e74a-42e0-a2f8-6b3089b9a02e admin admin] PATH_INFO: `/v3/roles` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:31.303927 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574ef34-e74a-42e0-a2f8-6b3089b9a02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:31.305555 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574ef34-e74a-42e0-a2f8-6b3089b9a02e admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:31.318011 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1427/2882] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:33:31.322123 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ab497-9d38-4cc3-a093-0e8a9ae17e4b admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:31.332768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ab497-9d38-4cc3-a093-0e8a9ae17e4b admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:31.337500 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f086dd-8705-4a44-a2a4-b4ac8784f04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:31.338209 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f086dd-8705-4a44-a2a4-b4ac8784f04f admin admin] REQUEST_METHOD: `PUT` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:31.338489 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f086dd-8705-4a44-a2a4-b4ac8784f04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:31.338761 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f086dd-8705-4a44-a2a4-b4ac8784f04f admin admin] PATH_INFO: `/v3/projects/fc2ae5fda72a457a98c6a798cd20eddb/users/5ce1d0b292764dfca8a4b483d0729064/roles/8b2ee9b11a934b67aa3fd68c497fbdf7` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:31.354425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f086dd-8705-4a44-a2a4-b4ac8784f04f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2ae5fda72a457a98c6a798cd20eddb, user_id=5ce1d0b292764dfca8a4b483d0729064, role_id=8b2ee9b11a934b67aa3fd68c497fbdf7)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:31.357112 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f086dd-8705-4a44-a2a4-b4ac8784f04f admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:31.453481 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1456/2883] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:33:31 2022] PUT /identity/v3/projects/fc2ae5fda72a457a98c6a798cd20eddb/users/5ce1d0b292764dfca8a4b483d0729064/roles/8b2ee9b11a934b67aa3fd68c497fbdf7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:31.465277 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323f122-2e18-4127-b3db-0916e672536f None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:31.465599 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323f122-2e18-4127-b3db-0916e672536f None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:31.465900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323f122-2e18-4127-b3db-0916e672536f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:31.493519 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-c323f122-2e18-4127-b3db-0916e672536f None None] MFA Rules not processed for user `5ce1d0b292764dfca8a4b483d0729064`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:31.494714 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-c323f122-2e18-4127-b3db-0916e672536f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:31.513490 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:31.558164 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1428/2884] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:31.563646 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f086dd-8705-4a44-a2a4-b4ac8784f04f admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:31.596895 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:31.624793 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f086dd-8705-4a44-a2a4-b4ac8784f04f admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:31.629448 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829300f-ba83-4bab-b3b5-6c8708bfda65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:31.630045 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829300f-ba83-4bab-b3b5-6c8708bfda65 admin admin] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:31.630330 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829300f-ba83-4bab-b3b5-6c8708bfda65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:31.630603 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829300f-ba83-4bab-b3b5-6c8708bfda65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:31.631238 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829300f-ba83-4bab-b3b5-6c8708bfda65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:31.633153 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829300f-ba83-4bab-b3b5-6c8708bfda65 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:31.734992 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1457/2885] 10.222.0.27 () {60 vars in 1153 bytes} [Fri Aug 12 12:33:31 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 12 12:33:31.739028 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323f122-2e18-4127-b3db-0916e672536f None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:31.750513 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323f122-2e18-4127-b3db-0916e672536f None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:31.806986 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b39662-4818-4261-b622-ab80f5600df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:31.807593 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b39662-4818-4261-b622-ab80f5600df5 admin admin] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:31.807875 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b39662-4818-4261-b622-ab80f5600df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:31.808153 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b39662-4818-4261-b622-ab80f5600df5 admin admin] PATH_INFO: `/v3/users` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:31.808848 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b39662-4818-4261-b622-ab80f5600df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:31.811208 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b39662-4818-4261-b622-ab80f5600df5 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:31.899854 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1429/2886] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 12 12:33:31 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 12 12:33:31.904580 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829300f-ba83-4bab-b3b5-6c8708bfda65 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:31.914917 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829300f-ba83-4bab-b3b5-6c8708bfda65 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:31.920100 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf15c24-666e-408d-b187-059d6f48e6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:31.920800 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf15c24-666e-408d-b187-059d6f48e6cb admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:31.921082 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf15c24-666e-408d-b187-059d6f48e6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:31.921351 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf15c24-666e-408d-b187-059d6f48e6cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:31.922026 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf15c24-666e-408d-b187-059d6f48e6cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:31.923587 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf15c24-666e-408d-b187-059d6f48e6cb admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:31.933580 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1458/2887] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:33:31.939250 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b39662-4818-4261-b622-ab80f5600df5 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:31.949332 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b39662-4818-4261-b622-ab80f5600df5 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:31.953960 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ef782-b866-41f7-a5d5-88c9b337e973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:31.954693 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ef782-b866-41f7-a5d5-88c9b337e973 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:31.954971 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ef782-b866-41f7-a5d5-88c9b337e973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:31.955245 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ef782-b866-41f7-a5d5-88c9b337e973 admin admin] PATH_INFO: `/v3/projects/20a02e660671426e98a9fb26ff85992f/users/d56dd32e946a44ca96a9e81742295015/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:31.970648 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ef782-b866-41f7-a5d5-88c9b337e973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a02e660671426e98a9fb26ff85992f, user_id=d56dd32e946a44ca96a9e81742295015, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:31.973380 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ef782-b866-41f7-a5d5-88c9b337e973 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:32.065879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1430/2888] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:33:31 2022] PUT /identity/v3/projects/20a02e660671426e98a9fb26ff85992f/users/d56dd32e946a44ca96a9e81742295015/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:32.069960 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf15c24-666e-408d-b187-059d6f48e6cb admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:32.080131 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf15c24-666e-408d-b187-059d6f48e6cb admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:32.135379 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25e8c5-aa4d-4841-b9e6-087302826dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:32.135905 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25e8c5-aa4d-4841-b9e6-087302826dbb admin admin] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:32.136186 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25e8c5-aa4d-4841-b9e6-087302826dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:32.136458 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25e8c5-aa4d-4841-b9e6-087302826dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:32.137147 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25e8c5-aa4d-4841-b9e6-087302826dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:32.139320 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25e8c5-aa4d-4841-b9e6-087302826dbb admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:32.148727 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1459/2889] 10.222.0.27 () {58 vars in 1122 bytes} [Fri Aug 12 12:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 12 12:33:32.153623 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ef782-b866-41f7-a5d5-88c9b337e973 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:32.163637 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ef782-b866-41f7-a5d5-88c9b337e973 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:32.168175 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961411cd-bed0-4636-b2de-4151fab00e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:32.168838 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961411cd-bed0-4636-b2de-4151fab00e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:32.169117 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961411cd-bed0-4636-b2de-4151fab00e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:32.169385 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961411cd-bed0-4636-b2de-4151fab00e60 admin admin] PATH_INFO: `/v3/projects/20a02e660671426e98a9fb26ff85992f/users/d56dd32e946a44ca96a9e81742295015/roles/906e3a8a6c7540a38f629f2567133347` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:32.171268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961411cd-bed0-4636-b2de-4151fab00e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a02e660671426e98a9fb26ff85992f, user_id=d56dd32e946a44ca96a9e81742295015, role_id=906e3a8a6c7540a38f629f2567133347)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:32.173896 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961411cd-bed0-4636-b2de-4151fab00e60 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:32.183611 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1431/2890] 10.222.0.27 () {60 vars in 1483 bytes} [Fri Aug 12 12:33:32 2022] PUT /identity/v3/projects/20a02e660671426e98a9fb26ff85992f/users/d56dd32e946a44ca96a9e81742295015/roles/906e3a8a6c7540a38f629f2567133347 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:32.193792 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bddb8a-ecf2-4b65-ba25-08edf7dde193 None None] REQUEST_METHOD: `POST` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:32.194105 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bddb8a-ecf2-4b65-ba25-08edf7dde193 None None] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:32.194391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bddb8a-ecf2-4b65-ba25-08edf7dde193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:32.222965 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.auth.core [None req-07bddb8a-ecf2-4b65-ba25-08edf7dde193 None None] MFA Rules not processed for user `d56dd32e946a44ca96a9e81742295015`. Rule list: `[]` (Enabled: `True`). {{(pid=81794) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:32.224214 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-07bddb8a-ecf2-4b65-ba25-08edf7dde193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:32.290062 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1460/2891] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:32.302010 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961411cd-bed0-4636-b2de-4151fab00e60 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:32.314438 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-961411cd-bed0-4636-b2de-4151fab00e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:32.361679 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961411cd-bed0-4636-b2de-4151fab00e60 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:32.365347 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff583f-a28d-41bf-bf40-66b79fbeca5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '935ab3a263ae46fcb929029a147a1010', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:32.365943 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff583f-a28d-41bf-bf40-66b79fbeca5b service cinder] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:32.366222 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff583f-a28d-41bf-bf40-66b79fbeca5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:32.366495 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff583f-a28d-41bf-bf40-66b79fbeca5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:32.375617 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff583f-a28d-41bf-bf40-66b79fbeca5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:32.377045 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff583f-a28d-41bf-bf40-66b79fbeca5b service cinder] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:32.386528 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1432/2892] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:33:32.416479 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bddb8a-ecf2-4b65-ba25-08edf7dde193 None None] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:32.445492 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:32.478252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bddb8a-ecf2-4b65-ba25-08edf7dde193 None None] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:32.483110 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a1117-52a9-41c9-8dab-581095345376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:32.487915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a1117-52a9-41c9-8dab-581095345376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:32.487915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a1117-52a9-41c9-8dab-581095345376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:32.487915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a1117-52a9-41c9-8dab-581095345376 admin admin] PATH_INFO: `/v3/users/5ce1d0b292764dfca8a4b483d0729064` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:32.487915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a1117-52a9-41c9-8dab-581095345376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce1d0b292764dfca8a4b483d0729064)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:32.487915 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a1117-52a9-41c9-8dab-581095345376 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:32.674003 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-922a1117-52a9-41c9-8dab-581095345376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce1d0b292764dfca8a4b483d0729064'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:32.738881 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-922a1117-52a9-41c9-8dab-581095345376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce1d0b292764dfca8a4b483d0729064'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:32.739566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-922a1117-52a9-41c9-8dab-581095345376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce1d0b292764dfca8a4b483d0729064'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:32.746861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-922a1117-52a9-41c9-8dab-581095345376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce1d0b292764dfca8a4b483d0729064'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:32.759075 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-922a1117-52a9-41c9-8dab-581095345376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce1d0b292764dfca8a4b483d0729064'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:32.765055 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1461/2893] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:33:32 2022] DELETE /identity/v3/users/5ce1d0b292764dfca8a4b483d0729064 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:32.769557 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5a214-5357-48d9-ba32-aff6eb4b61d9 None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:32.778532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5a214-5357-48d9-ba32-aff6eb4b61d9 None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:32.778532 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5a214-5357-48d9-ba32-aff6eb4b61d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:32.788995 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-76a5a214-5357-48d9-ba32-aff6eb4b61d9 None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:32.790359 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-76a5a214-5357-48d9-ba32-aff6eb4b61d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:32.799981 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:32.850227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1433/2894] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 12 12:33:32.860898 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a1117-52a9-41c9-8dab-581095345376 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:32.862318 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-922a1117-52a9-41c9-8dab-581095345376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:32.919046 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a1117-52a9-41c9-8dab-581095345376 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:32.922742 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0cbac-4d98-4bf5-b773-a1d8f366ab90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:32.923343 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0cbac-4d98-4bf5-b773-a1d8f366ab90 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:32.923627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0cbac-4d98-4bf5-b773-a1d8f366ab90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:32.923901 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0cbac-4d98-4bf5-b773-a1d8f366ab90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:32.934331 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0cbac-4d98-4bf5-b773-a1d8f366ab90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:32.935782 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0cbac-4d98-4bf5-b773-a1d8f366ab90 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:32.945984 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1462/2895] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:33:33.334405 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5a214-5357-48d9-ba32-aff6eb4b61d9 None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:33.336735 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-76a5a214-5357-48d9-ba32-aff6eb4b61d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:33.352001 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5a214-5357-48d9-ba32-aff6eb4b61d9 None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:33.357488 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:33.358173 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:33.358457 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:33.358737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce admin admin] PATH_INFO: `/v3/projects/fc2ae5fda72a457a98c6a798cd20eddb` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:33.359881 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc2ae5fda72a457a98c6a798cd20eddb)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:33.361825 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:33.478977 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc2ae5fda72a457a98c6a798cd20eddb', 'request_id': 'req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:33.485752 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc2ae5fda72a457a98c6a798cd20eddb', 'request_id': 'req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:33.492037 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc2ae5fda72a457a98c6a798cd20eddb', 'request_id': 'req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:33.562340 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce admin admin] The token cache is being invalidate because project fc2ae5fda72a457a98c6a798cd20eddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:33:33.562616 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:33.563249 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:33.565467 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1434/2896] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:33:33 2022] DELETE /identity/v3/projects/fc2ae5fda72a457a98c6a798cd20eddb => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:33.568920 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0cbac-4d98-4bf5-b773-a1d8f366ab90 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:33.579261 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0cbac-4d98-4bf5-b773-a1d8f366ab90 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:33.640444 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:33.652504 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710188e-39eb-4cce-b396-b70c5cc50057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:33.653866 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710188e-39eb-4cce-b396-b70c5cc50057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:33.654196 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710188e-39eb-4cce-b396-b70c5cc50057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:33.654475 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710188e-39eb-4cce-b396-b70c5cc50057 admin admin] PATH_INFO: `/v3/users/d56dd32e946a44ca96a9e81742295015` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:33.655627 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710188e-39eb-4cce-b396-b70c5cc50057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d56dd32e946a44ca96a9e81742295015)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:33.657602 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710188e-39eb-4cce-b396-b70c5cc50057 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:33.845426 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c710188e-39eb-4cce-b396-b70c5cc50057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd56dd32e946a44ca96a9e81742295015'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:33.905625 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c710188e-39eb-4cce-b396-b70c5cc50057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd56dd32e946a44ca96a9e81742295015'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:33.938042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c710188e-39eb-4cce-b396-b70c5cc50057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd56dd32e946a44ca96a9e81742295015'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:33.938042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c710188e-39eb-4cce-b396-b70c5cc50057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd56dd32e946a44ca96a9e81742295015'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:33.938042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-c710188e-39eb-4cce-b396-b70c5cc50057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd56dd32e946a44ca96a9e81742295015'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:33.938042 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1463/2897] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:33:33 2022] DELETE /identity/v3/users/d56dd32e946a44ca96a9e81742295015 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:34.394180 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:34.395819 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:34.414241 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484aca9-c3f9-4617-8d36-fb9ae0b315ce admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:34.475925 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0398561-db01-49f5-914f-f5f828147630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:34.476547 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0398561-db01-49f5-914f-f5f828147630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:34.476856 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0398561-db01-49f5-914f-f5f828147630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:34.477114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0398561-db01-49f5-914f-f5f828147630 admin admin] PATH_INFO: `/v3/projects/20a02e660671426e98a9fb26ff85992f` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:34.478225 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0398561-db01-49f5-914f-f5f828147630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20a02e660671426e98a9fb26ff85992f)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:34.480114 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0398561-db01-49f5-914f-f5f828147630 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:34.580498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:34.586231 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d0398561-db01-49f5-914f-f5f828147630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a02e660671426e98a9fb26ff85992f', 'request_id': 'req-d0398561-db01-49f5-914f-f5f828147630'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:34.592049 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d0398561-db01-49f5-914f-f5f828147630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a02e660671426e98a9fb26ff85992f', 'request_id': 'req-d0398561-db01-49f5-914f-f5f828147630'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:34.598168 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d0398561-db01-49f5-914f-f5f828147630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a02e660671426e98a9fb26ff85992f', 'request_id': 'req-d0398561-db01-49f5-914f-f5f828147630'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:34.681811 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d0398561-db01-49f5-914f-f5f828147630 admin admin] The token cache is being invalidate because project 20a02e660671426e98a9fb26ff85992f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:33:34.682100 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d0398561-db01-49f5-914f-f5f828147630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:34.682722 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-d0398561-db01-49f5-914f-f5f828147630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:34.684876 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1435/2898] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:33:34 2022] DELETE /identity/v3/projects/20a02e660671426e98a9fb26ff85992f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:41.076023 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710188e-39eb-4cce-b396-b70c5cc50057 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:41.077491 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-c710188e-39eb-4cce-b396-b70c5cc50057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:41.091197 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710188e-39eb-4cce-b396-b70c5cc50057 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:41.146596 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78835ab2-f541-4c00-9cf2-9feb6ed29b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:41.147219 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78835ab2-f541-4c00-9cf2-9feb6ed29b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:41.147503 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78835ab2-f541-4c00-9cf2-9feb6ed29b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:41.147780 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78835ab2-f541-4c00-9cf2-9feb6ed29b90 admin admin] PATH_INFO: `/v3/users/563ee9b635854f3fa70eae9564d14554` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:41.148949 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78835ab2-f541-4c00-9cf2-9feb6ed29b90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=563ee9b635854f3fa70eae9564d14554)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:41.150859 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78835ab2-f541-4c00-9cf2-9feb6ed29b90 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:41.312205 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-78835ab2-f541-4c00-9cf2-9feb6ed29b90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '563ee9b635854f3fa70eae9564d14554'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:41.373051 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-78835ab2-f541-4c00-9cf2-9feb6ed29b90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '563ee9b635854f3fa70eae9564d14554'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:41.373601 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-78835ab2-f541-4c00-9cf2-9feb6ed29b90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '563ee9b635854f3fa70eae9564d14554'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:41.380883 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-78835ab2-f541-4c00-9cf2-9feb6ed29b90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '563ee9b635854f3fa70eae9564d14554'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:41.390660 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:41.393381 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-78835ab2-f541-4c00-9cf2-9feb6ed29b90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '563ee9b635854f3fa70eae9564d14554'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:41.399278 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1464/2899] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:33:41 2022] DELETE /identity/v3/users/563ee9b635854f3fa70eae9564d14554 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:41.403867 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb54a0-a568-4424-ad5c-0ea9b4c4d83c None None] REQUEST_METHOD: `POST` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:41.404421 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb54a0-a568-4424-ad5c-0ea9b4c4d83c None None] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:41.404548 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb54a0-a568-4424-ad5c-0ea9b4c4d83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:41.425163 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.auth.core [None req-86eb54a0-a568-4424-ad5c-0ea9b4c4d83c None None] MFA Rules not processed for user `1b637f155b514b58ab1c4d9c02381dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81793) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 12 12:33:41.426362 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-86eb54a0-a568-4424-ad5c-0ea9b4c4d83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:41.485319 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1436/2900] 10.222.0.27 () {58 vars in 958 bytes} [Fri Aug 12 12:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 12 12:33:41.495149 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78835ab2-f541-4c00-9cf2-9feb6ed29b90 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:41.496581 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-78835ab2-f541-4c00-9cf2-9feb6ed29b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:41.553140 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78835ab2-f541-4c00-9cf2-9feb6ed29b90 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:41.556813 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2d0f5-0f4a-482a-a233-6be9b3170988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:41.557422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2d0f5-0f4a-482a-a233-6be9b3170988 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:41.557706 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2d0f5-0f4a-482a-a233-6be9b3170988 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:41.557981 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2d0f5-0f4a-482a-a233-6be9b3170988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:41.567460 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2d0f5-0f4a-482a-a233-6be9b3170988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:41.568913 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2d0f5-0f4a-482a-a233-6be9b3170988 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:41.578259 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1465/2901] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:33:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:33:42.009836 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb54a0-a568-4424-ad5c-0ea9b4c4d83c None None] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:33:42.011391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-86eb54a0-a568-4424-ad5c-0ea9b4c4d83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:33:42.025472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb54a0-a568-4424-ad5c-0ea9b4c4d83c None None] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:33:42.030089 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:33:42.030712 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:33:42.031002 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:33:42.031275 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd admin admin] PATH_INFO: `/v3/projects/a194328bf98443bc9e6d2b337a8233d7` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:33:42.037801 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a194328bf98443bc9e6d2b337a8233d7)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:33:42.037801 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:33:42.138899 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a194328bf98443bc9e6d2b337a8233d7', 'request_id': 'req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:42.144774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a194328bf98443bc9e6d2b337a8233d7', 'request_id': 'req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:42.150900 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a194328bf98443bc9e6d2b337a8233d7', 'request_id': 'req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:42.217979 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd admin admin] The token cache is being invalidate because project a194328bf98443bc9e6d2b337a8233d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81793) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:33:42.218268 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:42.218879 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:33:42.220459 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1437/2902] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:33:42 2022] DELETE /identity/v3/projects/a194328bf98443bc9e6d2b337a8233d7 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:33:51.574036 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=31,keystone:DELETE=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:33:52.215031 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=63,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:35:08.737290 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2d0f5-0f4a-482a-a233-6be9b3170988 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:35:08.745888 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2d0f5-0f4a-482a-a233-6be9b3170988 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:35:08.793364 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf833b13-acf8-4c9b-8b23-61e7a7ea87b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:35:08.794566 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf833b13-acf8-4c9b-8b23-61e7a7ea87b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:35:08.794929 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf833b13-acf8-4c9b-8b23-61e7a7ea87b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:35:08.795202 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf833b13-acf8-4c9b-8b23-61e7a7ea87b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:35:08.796986 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-cf833b13-acf8-4c9b-8b23-61e7a7ea87b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:35:08.806636 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf833b13-acf8-4c9b-8b23-61e7a7ea87b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:35:08.808059 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf833b13-acf8-4c9b-8b23-61e7a7ea87b8 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:35:08.817009 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1466/2903] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:35:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:35:15.997776 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:35:15.999467 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:35:16.012058 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7031ee-e7c7-4c8f-b394-e3a2296a30dd admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:35:16.016550 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021d9be-a704-426e-b418-4145de12c0b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:35:16.017192 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021d9be-a704-426e-b418-4145de12c0b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:35:16.017472 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021d9be-a704-426e-b418-4145de12c0b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:35:16.017745 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021d9be-a704-426e-b418-4145de12c0b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:35:16.019549 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8021d9be-a704-426e-b418-4145de12c0b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:35:16.093573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021d9be-a704-426e-b418-4145de12c0b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:35:16.095013 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8021d9be-a704-426e-b418-4145de12c0b4 service neutron] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:35:16.106176 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1438/2904] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:35:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:35:17.285759 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf833b13-acf8-4c9b-8b23-61e7a7ea87b8 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:35:17.295548 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf833b13-acf8-4c9b-8b23-61e7a7ea87b8 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:35:17.299465 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f524c04-b473-4372-9f23-e86c329ca781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:35:17.300074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f524c04-b473-4372-9f23-e86c329ca781 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:35:17.300359 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f524c04-b473-4372-9f23-e86c329ca781 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:35:17.300634 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f524c04-b473-4372-9f23-e86c329ca781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:35:17.302848 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2f524c04-b473-4372-9f23-e86c329ca781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:35:17.317994 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f524c04-b473-4372-9f23-e86c329ca781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:35:17.319449 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f524c04-b473-4372-9f23-e86c329ca781 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:35:17.331212 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1467/2905] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:35:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:35:18.577505 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021d9be-a704-426e-b418-4145de12c0b4 service neutron] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:35:18.578953 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-8021d9be-a704-426e-b418-4145de12c0b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:35:18.593189 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021d9be-a704-426e-b418-4145de12c0b4 service neutron] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:35:18.598727 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd0b0b-1574-48a4-bbef-93e56a11ddda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '687d0375abd44d2481f988c6df90fad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:35:18.599333 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd0b0b-1574-48a4-bbef-93e56a11ddda service nova] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:35:18.599615 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd0b0b-1574-48a4-bbef-93e56a11ddda service nova] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:35:18.599886 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd0b0b-1574-48a4-bbef-93e56a11ddda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:35:18.601737 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d0fd0b0b-1574-48a4-bbef-93e56a11ddda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:35:18.614819 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd0b0b-1574-48a4-bbef-93e56a11ddda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:35:18.616463 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd0b0b-1574-48a4-bbef-93e56a11ddda service nova] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:35:18.627842 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1439/2906] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:35:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:35:27.325927 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=32 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:35:28.623478 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=40 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:35:30.526185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f524c04-b473-4372-9f23-e86c329ca781 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:35:30.535245 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f524c04-b473-4372-9f23-e86c329ca781 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:35:30.538857 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec475f8-ebfd-4853-8c1e-48ce45950723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '745fad9c109a4280906eb0349849117d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:35:30.539447 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec475f8-ebfd-4853-8c1e-48ce45950723 service neutron] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:35:30.539731 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec475f8-ebfd-4853-8c1e-48ce45950723 service neutron] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:35:30.540007 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec475f8-ebfd-4853-8c1e-48ce45950723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:35:30.541841 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2ec475f8-ebfd-4853-8c1e-48ce45950723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:35:30.605248 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec475f8-ebfd-4853-8c1e-48ce45950723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:35:30.606710 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec475f8-ebfd-4853-8c1e-48ce45950723 service neutron] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:35:30.616768 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1468/2907] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:35:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:35:33.910773 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd0b0b-1574-48a4-bbef-93e56a11ddda service nova] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:35:33.912256 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-d0fd0b0b-1574-48a4-bbef-93e56a11ddda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:35:33.982584 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd0b0b-1574-48a4-bbef-93e56a11ddda service nova] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:35:33.988261 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db85515-646b-4280-b70d-b9fa7d46ca2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ee8ba21bc184feeaafacab3e69e87bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:35:33.988899 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db85515-646b-4280-b70d-b9fa7d46ca2b service placement] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:35:33.989187 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db85515-646b-4280-b70d-b9fa7d46ca2b service placement] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:35:33.989466 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db85515-646b-4280-b70d-b9fa7d46ca2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:35:33.991183 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9db85515-646b-4280-b70d-b9fa7d46ca2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:35:34.003416 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db85515-646b-4280-b70d-b9fa7d46ca2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:35:34.005068 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db85515-646b-4280-b70d-b9fa7d46ca2b service placement] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:35:34.014808 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1440/2908] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:35:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:35:37.526718 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec475f8-ebfd-4853-8c1e-48ce45950723 service neutron] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:35:37.539720 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec475f8-ebfd-4853-8c1e-48ce45950723 service neutron] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:35:37.545391 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da0c61-7031-4075-ab7a-afeb211e44f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ee8ba21bc184feeaafacab3e69e87bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:35:37.546216 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da0c61-7031-4075-ab7a-afeb211e44f4 service placement] REQUEST_METHOD: `GET` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:35:37.546498 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da0c61-7031-4075-ab7a-afeb211e44f4 service placement] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:35:37.546855 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da0c61-7031-4075-ab7a-afeb211e44f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:35:37.549393 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-04da0c61-7031-4075-ab7a-afeb211e44f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:35:37.561689 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da0c61-7031-4075-ab7a-afeb211e44f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:35:37.563232 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da0c61-7031-4075-ab7a-afeb211e44f4 service placement] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:35:37.573349 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1469/2909] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:35:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:35:37.634677 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db85515-646b-4280-b70d-b9fa7d46ca2b service placement] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:35:37.636112 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-9db85515-646b-4280-b70d-b9fa7d46ca2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:35:37.650846 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db85515-646b-4280-b70d-b9fa7d46ca2b service placement] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:35:37.655501 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f731cd-48c7-4eb6-a3b9-b6b878a69ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:35:37.656121 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f731cd-48c7-4eb6-a3b9-b6b878a69ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:35:37.657111 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f731cd-48c7-4eb6-a3b9-b6b878a69ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:35:37.657398 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f731cd-48c7-4eb6-a3b9-b6b878a69ac5 admin admin] PATH_INFO: `/v3/users/a90e3591b4144fdfb8b5ed5391aec7c6` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:35:37.658638 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f731cd-48c7-4eb6-a3b9-b6b878a69ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a90e3591b4144fdfb8b5ed5391aec7c6)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:35:37.660560 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f731cd-48c7-4eb6-a3b9-b6b878a69ac5 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:35:37.847473 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-93f731cd-48c7-4eb6-a3b9-b6b878a69ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a90e3591b4144fdfb8b5ed5391aec7c6'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:37.848227 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-93f731cd-48c7-4eb6-a3b9-b6b878a69ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a90e3591b4144fdfb8b5ed5391aec7c6'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:37.919191 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-93f731cd-48c7-4eb6-a3b9-b6b878a69ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a90e3591b4144fdfb8b5ed5391aec7c6'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:37.932000 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-93f731cd-48c7-4eb6-a3b9-b6b878a69ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a90e3591b4144fdfb8b5ed5391aec7c6'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:37.939695 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-93f731cd-48c7-4eb6-a3b9-b6b878a69ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a90e3591b4144fdfb8b5ed5391aec7c6'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:37.945635 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1441/2910] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:35:37 2022] DELETE /identity/v3/users/a90e3591b4144fdfb8b5ed5391aec7c6 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:35:38.195010 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da0c61-7031-4075-ab7a-afeb211e44f4 service placement] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:35:38.207336 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da0c61-7031-4075-ab7a-afeb211e44f4 service placement] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:35:38.264669 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e142e4b-adab-42df-9b2a-4e28692f0ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:35:38.265302 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e142e4b-adab-42df-9b2a-4e28692f0ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:35:38.265585 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e142e4b-adab-42df-9b2a-4e28692f0ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:35:38.265863 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e142e4b-adab-42df-9b2a-4e28692f0ce4 admin admin] PATH_INFO: `/v3/projects/566d2428ccbe41888b9619faf5f212a2` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:35:38.266970 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e142e4b-adab-42df-9b2a-4e28692f0ce4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=566d2428ccbe41888b9619faf5f212a2)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:35:38.268896 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e142e4b-adab-42df-9b2a-4e28692f0ce4 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:35:38.353861 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2e142e4b-adab-42df-9b2a-4e28692f0ce4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '566d2428ccbe41888b9619faf5f212a2', 'request_id': 'req-2e142e4b-adab-42df-9b2a-4e28692f0ce4'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:38.359421 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2e142e4b-adab-42df-9b2a-4e28692f0ce4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '566d2428ccbe41888b9619faf5f212a2', 'request_id': 'req-2e142e4b-adab-42df-9b2a-4e28692f0ce4'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:38.365740 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2e142e4b-adab-42df-9b2a-4e28692f0ce4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '566d2428ccbe41888b9619faf5f212a2', 'request_id': 'req-2e142e4b-adab-42df-9b2a-4e28692f0ce4'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:38.432624 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2e142e4b-adab-42df-9b2a-4e28692f0ce4 admin admin] The token cache is being invalidate because project 566d2428ccbe41888b9619faf5f212a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:35:38.432944 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2e142e4b-adab-42df-9b2a-4e28692f0ce4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:38.433567 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-2e142e4b-adab-42df-9b2a-4e28692f0ce4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:38.435056 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1470/2911] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:35:38 2022] DELETE /identity/v3/projects/566d2428ccbe41888b9619faf5f212a2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:35:38.445566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f731cd-48c7-4eb6-a3b9-b6b878a69ac5 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:35:38.445566 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-93f731cd-48c7-4eb6-a3b9-b6b878a69ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:35:38.454470 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f731cd-48c7-4eb6-a3b9-b6b878a69ac5 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:35:38.511121 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eae66a-da18-4a8c-8735-5e4d495fc252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:35:38.511774 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eae66a-da18-4a8c-8735-5e4d495fc252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:35:38.512059 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eae66a-da18-4a8c-8735-5e4d495fc252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:35:38.512335 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eae66a-da18-4a8c-8735-5e4d495fc252 admin admin] PATH_INFO: `/v3/users/9d2d4d28f99d4661b9cdfaa7832c4e2a` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:35:38.513541 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eae66a-da18-4a8c-8735-5e4d495fc252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d2d4d28f99d4661b9cdfaa7832c4e2a)` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:35:38.515498 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eae66a-da18-4a8c-8735-5e4d495fc252 admin admin] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:35:38.518573 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:35:38.742242 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-17eae66a-da18-4a8c-8735-5e4d495fc252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d2d4d28f99d4661b9cdfaa7832c4e2a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:38.742898 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-17eae66a-da18-4a8c-8735-5e4d495fc252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d2d4d28f99d4661b9cdfaa7832c4e2a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:38.839391 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-17eae66a-da18-4a8c-8735-5e4d495fc252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d2d4d28f99d4661b9cdfaa7832c4e2a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:38.851533 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-17eae66a-da18-4a8c-8735-5e4d495fc252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d2d4d28f99d4661b9cdfaa7832c4e2a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:38.858195 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.notifications [None req-17eae66a-da18-4a8c-8735-5e4d495fc252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d2d4d28f99d4661b9cdfaa7832c4e2a'} {{(pid=81793) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:38.864037 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1442/2912] 10.222.0.27 () {58 vars in 1224 bytes} [Fri Aug 12 12:35:38 2022] DELETE /identity/v3/users/9d2d4d28f99d4661b9cdfaa7832c4e2a => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:35:39.142684 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e142e4b-adab-42df-9b2a-4e28692f0ce4 admin admin] Authenticating user token {{(pid=81794) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:35:39.144164 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.fernet_utils [None req-2e142e4b-adab-42df-9b2a-4e28692f0ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81794) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:35:39.157769 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e142e4b-adab-42df-9b2a-4e28692f0ce4 admin admin] Validating token access rules against request {{(pid=81794) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:35:39.166296 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:35:39.215252 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcd711-e527-4ea7-8114-327906626867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b637f155b514b58ab1c4d9c02381dc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e93c7bb262f46249ca80b95c794bf2f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81794) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:35:39.215873 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcd711-e527-4ea7-8114-327906626867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:35:39.216151 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcd711-e527-4ea7-8114-327906626867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:35:39.216422 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcd711-e527-4ea7-8114-327906626867 admin admin] PATH_INFO: `/v3/projects/82ceaffd799d43c2ad2fc4d6e3289cf9` {{(pid=81794) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:35:39.217553 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dcd711-e527-4ea7-8114-327906626867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ceaffd799d43c2ad2fc4d6e3289cf9)` {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:35:39.219594 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dcd711-e527-4ea7-8114-327906626867 admin admin] RBAC: Authorization granted {{(pid=81794) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:35:39.330185 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-64dcd711-e527-4ea7-8114-327906626867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ceaffd799d43c2ad2fc4d6e3289cf9', 'request_id': 'req-64dcd711-e527-4ea7-8114-327906626867'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:39.335648 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-64dcd711-e527-4ea7-8114-327906626867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ceaffd799d43c2ad2fc4d6e3289cf9', 'request_id': 'req-64dcd711-e527-4ea7-8114-327906626867'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:39.341977 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-64dcd711-e527-4ea7-8114-327906626867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ceaffd799d43c2ad2fc4d6e3289cf9', 'request_id': 'req-64dcd711-e527-4ea7-8114-327906626867'} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:39.426425 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-64dcd711-e527-4ea7-8114-327906626867 admin admin] The token cache is being invalidate because project 82ceaffd799d43c2ad2fc4d6e3289cf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81794) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 12 12:35:39.426839 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-64dcd711-e527-4ea7-8114-327906626867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:39.427512 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG keystone.notifications [None req-64dcd711-e527-4ea7-8114-327906626867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81794) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 12 12:35:39.429074 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: [pid: 81794|app: 0|req: 1471/2913] 10.222.0.27 () {58 vars in 1233 bytes} [Fri Aug 12 12:35:39 2022] DELETE /identity/v3/projects/82ceaffd799d43c2ad2fc4d6e3289cf9 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 12 12:35:48.948418 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:35:49.803552 dv-cinder-797970-6-iscsi devstack@keystone.service[81794]: DEBUG dbcounter [-] [81794] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81794) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 12 12:38:10.182981 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eae66a-da18-4a8c-8735-5e4d495fc252 admin admin] Authenticating user token {{(pid=81793) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 12 12:38:10.184398 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-17eae66a-da18-4a8c-8735-5e4d495fc252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:38:10.323874 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eae66a-da18-4a8c-8735-5e4d495fc252 admin admin] Validating token access rules against request {{(pid=81793) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 12 12:38:10.382401 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc4884-4e72-4e63-836f-5fd1f780987d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ee8ba21bc184feeaafacab3e69e87bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20c5570631a34c4ab3868eb1cd96e345', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81793) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 12 12:38:10.383036 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc4884-4e72-4e63-836f-5fd1f780987d service placement] REQUEST_METHOD: `GET` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 12 12:38:10.383315 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc4884-4e72-4e63-836f-5fd1f780987d service placement] SCRIPT_NAME: `/identity` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 12 12:38:10.383583 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc4884-4e72-4e63-836f-5fd1f780987d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81793) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 12 12:38:10.385323 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.fernet_utils [None req-b7cc4884-4e72-4e63-836f-5fd1f780987d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81793) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 12 12:38:10.396633 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc4884-4e72-4e63-836f-5fd1f780987d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 12 12:38:10.398211 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc4884-4e72-4e63-836f-5fd1f780987d service placement] RBAC: Authorization granted {{(pid=81793) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 12 12:38:10.407946 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: [pid: 81793|app: 0|req: 1443/2914] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Aug 12 12:38:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 12 12:38:20.403754 dv-cinder-797970-6-iscsi devstack@keystone.service[81793]: DEBUG dbcounter [-] [81793] Writing DB stats keystone:SELECT=89 {{(pid=81793) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}